Shortest Paths for Sub-riemannian Metrics on Rank-two Distributions

We study length-minimizing arcs in sub-Riemannian manifolds (M;E;G) whose metric G is de ned on a rank-two bracket-generating distribution E. It is well known that all length-minimizing arcs are extremals, and that these extremals are either \normal" or \abnormal." Normal extremals are locally optimal, in the sense that every su ciently short piece of such an extremal is a minimizer. The question whether every length-minimizer is a normal extremal remained open for several years, and was recently settled by R. Montgomery, who exhibited a counterexample. But Montgomery's geometric optimality proof depends heavily on special properties of his example and still leaves open the question whether abnormal minimizers are an exceptional phenomenon or a common occurrence. We present an analytic technique for proving local optimality of a large class of abnormal extremals that we call \regular." Our technique is based on (a) a \normal form theorem," stating that, locally, a regular abnormal extremal can always be put in a special form by a suitable change of coordinates, and (b) an inequality showing that, once a trajectory is in this special form, then local optimality follows. Using this approach we prove that regular abnormal extremals are locally optimal. If E satis es a mild additional restriction |valid in particular for all regular 2-dimensional distributions and for generic 2-dimensional distributions| then regular abnormal extremals are \typical" (in a sense made precise in the text), so our result implies that the abnormal minimizers are ubiquitous rather than exceptional. We also discuss some related issues, and in particular show, by means of an example, that a smooth abnormal extremal need not be locally optimal, even if in addition it belongs to the class |recently studied by Bryant and Hsu| of C1-rigid curves. Keywords: Sub-Riemannian manifolds, Geodesics, Hamiltonians, Abnormal Extremals. ix 1 Introduction The structure of sub-Riemannian minimizers and of the corresponding \geodesics" has recently attracted a great deal of attention (cf. [1], [5], [6], [7], [9], [10], [11], [12], [14], [17], [18], [19], [22], [23], [27], [28]; see Remark 1 below for a discussion of the use of the word \geodesic"), due to the delicate issues that arise because of the possibility of the existence of \abnormal" length-minimizing arcs. This phenomenon, well known in Optimal Control Theory, was not immediately recognized as possible in the more special case of subRiemannian geometry. For example, in 1986 it was stated, in [22], that all length-minimizing arcs for a sub-Riemannian manifold are characteristics of the associated Hamiltonian (i.e. \normal extremals," in our terminology). A proof was suggested for this result, relying on an application of the Pontryagin Maximum Principle from Optimal Control Theory. It turns out, however, that the Maximum Principle only makes it possible to draw the weaker conclusion that every minimizer is either a characteristic of the Hamiltonian (i.e. a normal extremal) or a member of another class of arcs known as \abnormal extremals." The possibility that a minimizer might be an abnormal extremal can easily be ruled out in the Riemannian case and, more generally, for the special class |introduced by R. Strichartz in [23]| of sub-Riemannian metrics de ned on \strongly bracket-generating" distributions, but for general sub-Riemannian metrics there is no obvious way to go beyond the necessary conditions of the Maximum Principle and exclude abnormal extremals. This left open the question whether there can exist sub-Riemannian minimizers that are not normal extremals (\strictly abnormal minimizers," in the terminology introduced below). The suggestion that this could indeed happen had in fact been made much Work supported in part by the National Science Foundation under NSF grant DMS92-02554. Received by the editor November 1993, and in revised form February 1994. 2 WENSHENG LIU AND H ECTOR J. SUSSMANN earlier, in 1977, by B. Gaveau, who had studied a particular subRiemannian structure for which he asserted that there were pairs of points p1, p2 that could not be joined by a characteristic curve ([9], p. 133, Theorem 1). Since, for the system studied by Gaveau, the existence of a minimizer joining any two points is obvious, his result would have amounted to a non-constructive proof of the existence of a strictly abnormal minimizer. However, contrary to the statement of [9], the points considered by Gaveau can be joined by a characteristic curve, as was pointed out by R. Brockett in 1984 (cf. [6]). (Brockett established this by explicitly computing the minimizers and showing that they were characteristics. A brief discussion of the Gaveau-Brockett system is included below in Appendix A, where we provide an alternative proof of Brockett's result, by directly applying the Pontryagin Maximum Principle to show that all minimizers are characteristic curves.) Meanwhile, the claim that all minimizers are normal extremals was made again in 1988 in [1]. Subsequently, relying on this assertion, Hamenst adt stated in 1990, in [12], that \the critical points are geodesics, i.e. locally minimizing curves," and \every geodesic is a critical point; together this gives a complete description of the geodesics." (Hamenst adt's \critical points" are exactly the same as our \normal extremals.") However, the claim of [1] is also incorrect, and the \complete description of the geodesics" announced in [12] is in fact incomplete, because it omits a rather important and interesting class of minimizers that are rather di erent from the normal extremals. Correct answers to these questions are in fact suggested in a rather natural way by Optimal Control Theory, which gives necessary conditions for trajectories of fairly general control systems to minimize functionals of a rather general type. The sub-Riemannian minimization problem just happens to be a special case of the much broader class of situations where the Maximum Principle of Optimal Control Theory applies. (This fact was recognized by Strichartz in [22], even SUB-RIEMANNIAN METRICS 3 though his result failed to include the abnormal extremals.) When the Maximum Principle is in fact applied to this case, the possibility that some minimizers might be \abnormal" emerges directly by mechanically writing the necessary conditions. This, of course, does not yet answer the question whether abnormal minimizers actually exist, but it provides a radically di erent perspective on the problem: whereas a di erential geometer's natural inclination is to look for the true characterization of minimizers in the form of a modi ed version of the geodesic equation, based on making variations as in the classical derivation of the Euler-Lagrange equations, the control theorist's predisposition is to start from the opposite direction, applying the Maximum Principle to conclude that a minimizer is either a \normal extremal" or an \abnormal" one, inferring from this that abnormal extremal minimizers probably exist, and then setting out to prove that they do. The problem of the existence of strictly abnormal minimizers was nally settled in 1991, when R. Montgomery, in [17], produced an example of such a minimizer, thereby showing that the intuition derived from the control-theoretic point of view was in fact the right one, even though Montgomery himself was led to his discovery by physical rather than control-theoretic considerations. Montgomery's very long and ingenious optimality proof was somewhat simpli ed in 1992 by I. Kupka (cf. [14]). However, neither of these proofs makes it possible to go beyond individual examples and prove, for instance, that large classes of abnormal extremals are optimal. In [16], we studied an example for which the optimality proof was much simpler. Moreover, this example had the extra virtue of being such that, after suitable changes of coordinates, one can transform fairly general situations into normal forms to which the proof of [16] applies. This was made precise in the preprint [25], where a general \optimality lemma" was announced, which basically describes the most general situation where a method similar to that of [16] can be applied to prove optimality. Using this optimality 4 WENSHENG LIU AND H ECTOR J. SUSSMANN lemma plus a coordinate transformation, it was proved in [25] that, for a completely arbitrary four-dimensional sub-Riemannian manifold whose metric is de ned on a regular two-dimensional distribution, there passes through each point exactly one locally simple abnormal extremal parametrized by arc-length, and all these abnormal extremals are locally optimal. (Here \simple" means \without double points.") It was also shown that, if an extra generic condition is satis ed, then these abnormal extremals are not normal. It then became apparent that abnormal minimizers are not pathological objects that can only be shown to exist in very special and elaborately constructed examples. They are in fact ubiquitous and easy to nd (at least for generic cases), and can be proved optimal by relatively simple general techniques. The purpose of this work is to present the general theory of regular abnormal extremals for manifolds M of arbitrary dimensions equipped with sub-Riemannian structures (E;G) arising from two-dimensional distributions, and to prove in particular that these abnormal extremals are locally optimal. (The fact that normal extremals are locally optimal is an immediate consequence of the Control Theory version of classical Hamilton-Jacobi theory, which gives su cient conditions for optimality of \ elds of extremals," as explained, for example, in [15]. An independent derivation was given by Hamenstadt in [12]. Since the proof of [12] is based on a formalism quite di erent from ours, and the control-theoretic proof is in our view a rather transparent illustration of the power of Optimal Control methods and the advantages of the Hamiltonian formulation, we have included the latter proof in Appendix C.)A special e ort has been made to produce a detailed, self-contained presentation. (We believe this is especially important in view of the fact that so many papers published on this subject contain incorrect statements.) We also insist on using the appropriate geometric language, which in our view is that of Optimal Control Theory, properly formuSUB-RIEMANNIAN METRICS 5 lated in a Hamiltonian setting. In particular, our starting point is the description of Riemannian geodesics as the projections of the bicharacteristics of a Hamiltonian function on the cotangent bundle, rather than the usual characterization in terms of the standard geodesic equation r _ _ = 0 in the tangent bundle. This formulation generalizes naturally to the sub-Riemannian case, and leads to a completely intrinsic de nition of \extremal." (This is to be contrasted, e.g., with the \simple di erential equation for the critical points" presented in [12], which evolves in the tangent bundle but depends on the choice of a frame complementary to the distribution.) To emphasize the coordinate-free nature of the characterization of extremals, we use a coordinate-free version of the Maximum Principle. Since the distribution E need not have a global basis of sections, we rely in fact on a version that is doubly intrinsic, in that it is not only coordinate-free but also formulated without any reference to choices of sections. Extremals are, by de nition, projections on M of certain arcs in the cotangent bundle T M of M called biextremals. Biextremals can be of either of two mutually exclusive types, namely, \normal" or \abnormal." An extremal is normal or abnormal depending on whether it is a projection of a normal or an abnormal biextremal. (An extremal can be both normal and abnormal. Abnormal extremals that are not normal will be called \strictly abnormal.") Normal extremals are obviously smooth, but abnormal extremals can fail to be smooth, or even of class C1 (cf. the example in Subsection 9.6.) The Pontryagin Maximum Principle implies that every arc that minimizes length is an extremal. As pointed out above, normal extremals locally minimize length, and it is then natural to ask whether abnormal extremals |or, perhaps, abnormal extremals that satisfy some extra conditions| also locally minimize. Two natural candidates for the \extra conditions" are smoothness and rigidity. (Rigid curves for rank 2 distributions were studied by R. Bryant and L. Hsu in [7]. The class of curves considered 6 WENSHENG LIU AND H ECTOR J. SUSSMANN here is almost exactly the same as that of [7]. For a discussion of the precise relationship between the two classes, cf. Remark 2 below.) We show, however, that an abnormal extremal need not be a local minimizer, even if it is both smooth and rigid. (This is done in Appendix D, where we exhibit a C1-rigid real-analytic abnormal extremal which is not a local minimizer.) So, abnormal extremals can be local minimizers, as in the examples of Montgomery and Kupka, but need not be. This naturally leads to the main question studied here, namely, which of the two possibilities occurs for \typical" abnormal extremals. Our answer |for distributions E of rank two| is that local minimization is the most common situation. We make this precise by rst de ning the class of \regular abnormal biextremals." Our main result is then Theorem 5, which says that regular abnormal extremals (i.e. projections of regular abnormal biextremals) locally minimize length. In addition, we show that, when the distribution E satis es a very mild extra condition, then these regular abnormal biextremals are \typical" in the following precise sense: if we use AEC(E) (the \abnormal extremal carrier of E) to denote the set of nonzero members of the annihilator of [E;E], then: (i) every nonconstant abnormal biextremal is contained in AEC(E), (ii) there is a relatively open dense subset RA(E) of AEC(E) which is a submanifold and carries a one-dimensional foliation FE whose leaves, properly parametrized, are the regular abnormal biextremals, (iii) every locally simple abnormal biextremal contained in RA(E) and parametrized by arc-length is regular. Remark 1 Arcs that minimize length are often called \geodesics." In the Riemannian case, the precise de nition of a \geodesic" is that it is a curve that satis es the necessary condition for length-minimization given by the geodesic equation or, equivalently, a curve that locally minimizes length. In the sub-Riemannian setting there is no unanimous SUB-RIEMANNIAN METRICS 7 agreement in the literature on the proper use of the word \geodesic." There is an obvious sub-Riemannian analogue of the geodesic equation, namely, the equation of non-null bicharacteristics of the Hamiltonian of the sub-Riemannian metric. This suggests the possibility of calling these bicharacteristics |or their projections on the base manifold| \geodesics." This would specialize in the Riemannian case to the usual de nition of geodesics, but in the more general sub-Riemannian setting it would no longer be true that every minimizer is a geodesic. Alternatively, we could choose to de ne a \geodesic" to be an arc that locally minimizes length. (Some authors |e.g. R. Montgomery in [17], whose rst version was pointedly entitled \Geodesics that do not satisfy the geodesic equations"| appear to be leaning in this direction, and Hamenstadt in [12] explicitly states that \geodesics" are \locally minimizing curves.") This would also reduce in the Riemannian case to the usual de nition of geodesics, and would in addition have the desired property that all minimizers are geodesics. On the other hand, the new de nition would have the drawback of making geodesics hard to characterize by means of di erential equations. (Such a characterization has not yet been found, and may very well not exist.) In this work we choose to avoid the word \geodesic" altogether. Arcs that minimize length are called \minimizers." Arcs that satisfy the necessary conditions for minimization given by the Maximum Principle of Optimal Control Theory are \extremals." Extremality is a necessary condition for an arc to be a minimizer, and has an explicit characterization in terms of constrained ordinary di erential equations, but in the sub-Riemannian case an extremal need not be a local minimizer, as explained above. Remark 2 In [7], Bryant and Hsu studied rigid curves for smooth distributions, and proved that, if a rank 2 distribution E on M satis es a nonintegrability condition, then through every point p of M there 8 WENSHENG LIU AND H ECTOR J. SUSSMANN passes a locally rigid curve. Our results are related to those of [7] as follows. The class of points p where our results apply is slightly larger than that of [7], because in [7] it is assumed that the distributions E and [E;E] are nowhere integrable, for which it is necessary that dim[E;E](p) = 3 and dim[E; [E;E]](p) > dim[E;E](p) (cf. [7], Theorem 3.1), whereas our results apply at every point p such that dim[E; [E;E]](p) > dim[E;E](p), even if dim[E;E](p) = 2. (This includes, in particular, the generic singularities of rank 2 distributions in dimension 3.) On the other hand, for the points considered in [7], our regular abnormal extremals are exactly the \projections of characteristics in Q 1" of [7]. The main result of [7] is that these curves are locally rigid. Our results |specialized to sub-Riemannian manifolds (M;E;G) for which E satis es the nonintegrability hypotheses of [7]| imply in particular that these curves are also local minimizers. In Appendix D we show that by using essentially the same method as in the proof of our local minimization theorem one can also establish the local rigidity of regular abnormal extremals for rank 2 distributions, thereby providing an alternative proof of the Bryant-Hsu result under our slightly more general conditions. Since the curves studied here and in [7] happen to be both locally rigid and locally minimizing, one might suspect that there is a deeper relationship between rigidity and local optimality, e.g. that perhaps the two properties are equivalent, or that one of them might imply the other. The purpose of Appendix D is to argue that this is not so, by showing that both implications are false in general. 2 Three examples Before discussing the general theory, we illustrate the main issues by means of three examples. The rst one is the classical theory of RieOur example of a nonrigid abnormal minimizer is for a rank 3 distribution. SUB-RIEMANNIAN METRICS 9 mannian geodesics, reformulated in the Hamiltonian language that generalizes naturally to the sub-Riemannian case. The second one is the simplest possible sub-Riemannian but not Riemannian situation, corresponding to the Heisenberg Lie algebra. In this case, the \natural" extension of the Riemannian theory of geodesics turns out to work, in the sense that the length-minimizing arcs are characteristic curves of the Hamiltonian associated to the metric. Finally, the third example | rst discussed in [16]| exhibits a case where the natural extension does not work: we explicitly show that a certain arc is a minimizer but is not a characteristic of the Hamiltonian. 2.1 Riemannian geodesics If M is a Riemannian manifold, with metric tensor G, then the curves that locally minimize length and are parametrized by arc-length are exactly those that satisfy, in local coordinates, the geodesic equation  xi + Pjk ijk _ xj _ xk = 0, where the ijk are the Christo el symbols of the Riemannian connection associated to the metric, together with the constraint Pi( _ xi)2 = 1. Alternatively, one can avoid the Christo el symbols by using the Legendre transformation to pass to the Hamiltonian formulation. This amounts to working on the cotangent bundle T M , and introducing a \momentum" covector whose components i are given by i = Pj gij(x) _ xj. (This is precisely |up to a signy| the familiar \lowering indices" operation.) We then de ne the Hamiltonian H : T M ! IR by letting H = 12 Pij gij(x) i j , i.e. H(x; ) = 1 2k k2G;x ; (1) where k kG;x is the norm of the covector , regarded as a linear functional on the tangent space TxM of M at x, endowed with the inner yOur somewhat nonstandard sign conventions are chosen so as to be consistent with those commonly used in Optimal Control Theory. 10 WENSHENG LIU AND H ECTOR J. SUSSMANN product G(x). The geodesic equations become _ x = @H @ , _ = @H @x , i.e. the usual Hamilton equations, whose solutions are the bicharacteristics of H. The constraint k _ xkG = 1 becomes the equality k kG = 1, i.e. H = 1 2. The projections on M of the bicharacteristics are the characteristics of H. In other words, Formula (1) associates to the Riemannian metric G a smooth real-valued function H on the cotangent bundle T M . The symplectic structure of T M associates to H a Hamiltonian vector eld !H . The projections on M of the integral curves of !H along which H 12 are the geodesics parametrized by arc-length. More generally, the geodesics parametrized by a constant times arc-length are precisely the projections on M of arbitrary nonnull bicharacteristics of H (i.e. integral curves of !H along which the constant value of H is 6= 0). 2.2 The Heisenberg algebra case We now discuss the simplest example of a sub-Riemannian structure which is not Riemannian. Let ! be the 1-form in IR3 given by ! = dz (x dy y dx) (2) We wish to consider arcs in IR3 that satisfy the velocity constraint h!; _ i = 0. Precisely, we let Ca;b be the set of all absolutely continuous curves : [a; b] ! IR3 that satisfy h!( (t)); _ (t)i = 0 for almost all t 2 [a; b], and de ne C = [ 1 0, then is called a cubic chart of radius . If p 2 U and (p) = (0; : : : ; 0), then we will say that is centered at p. For each chart = (x1; : : : ; xn) : U ! IRn on M there are induced charts T : 1 M (U) ! IR2n, T : ( M) 1(U) ! IR2n of TM , T M , given by T (x; v) = (x); (hdx1; vi; : : : ; hdxn; vi) ; T (x; ) = (x); (h ; @1i; : : : ; h ; @ni) ; where @jdef = @ @xj . A curve in a manifold M is a continuous map : I !M de ned on a subinterval I of the real line. An arc inM is an absolutely continuous curve whose domain is a compact interval. (\Absolute continuity" is of course well de ned in terms of local charts.) The boundary @ of an arc : [a; b]!M is the ordered pair @ = ( (a); (b)). 3.2 Hamiltonian functions, Hamilton vector elds, and bicharacteristics If N is a symplectic manifold with symplectic 2-form , and H 2 C1(N), we use ! H to denote the Hamilton vector eld associated to H. (By de nition, !H is the vector eld V on N such that (X;V ) = hdH;Xi for every vector eld X on N .) If H 2 Ck(N) and k 1, then the vector eld !H is of class Ck 1. If H;K 2 C1(N), then the Poisson bracket fH;Kg is the directional derivative of K in the 18 WENSHENG LIU AND H ECTOR J. SUSSMANN direction of !H , i.e. fH;Kg = hdK; !H i = ( !H ; ! K ). The well known formulas fH;KLg = fH;KgL+fH;LgK, fH; fK;Lgg+fK; fL;Hgg+ fL; fH;Kgg = 0 (the Jacobi identity) and ! HK = H ! K +K !H will be used throughout, as will the fact that the map H ! !H is a Lie algebra homomorphism from (C1(N); f ; g) to (V 1(N); [ ; ]). If H is any subset of C1(N), we de ne, for each x 2 N , a subset !H (x) of the tangent space TxN to N at x by letting !H (x) = f !H (x) : H 2 Hg. We then de ne a bicharacteristic of H to be an absolutely continuous curve on N such that _ (t) 2 !H ( (t)) for almost every t. Finally, de ne a null bicharacteristic of H to be a bicharacteristic along which all the functions H 2 H vanish. In the special case whenH consists of a single function H, the bicharacteristics (resp. null bicharacteristics) of H are called bicharacteristics (resp. null bicharacteristics) of H. Clearly, the bicharacteristics of a Hamiltonian H 2 Ck(N) are curves of class Ck along which H is constant. In particular, the bicharacteristics of a smooth Hamiltonian are smooth. If H C1(N) consists of more than one element, then in general the bicharacteristics of H need not be of class C1, even if H C1(N). If H1, H2 are linear spaces (over IR) of smooth functions on N , we call H1 and H2 equivalent if the C1(N)-modules generated by H1 and H2 coincide, i.e. if every element H of H2 is a nite linear combination f1H1+ : : :+fkHk of elements Hi 2 H1 with smooth coe cients fi, and viceversa. The following trivial observation will be used repeatedly: Proposition 2 If H1, H2 are equivalent linear spaces of smooth functions on a symplectic manifold N , then H1 and H2 have the same null bicharacteristics. 3.3 Hamiltonian lifts and characteristics The cotangent bundle T M of a manifold M has a natural symplectic structure determined by the 2-form M = d!M , where !M is the 1SUB-RIEMANNIAN METRICS 19 form given !M (x; )(v) = h ; d M (v)i for v 2 T(x; )(T M). Relative to a coordinate chart T = (x1; : : : ; xn; 1; : : : ; n) induced by a chart = (x1; : : : ; xn) on M , we have the formulas !M = Pj jdxj, M = Pj d j ^ dxj, !H = Pj @H @ j @ @xj @H @xj @ @ j , and fH;Kg = Pj @H @ j @K @xj @H @xj @K @ j . IfM is a smooth manifold and H C1(T M), then a characteristic (resp. null characteristic) of H is a curve on M which is of the form M o for some bicharacteristic (resp. null bicharacteristic) of H. If can be chosen to be entirely contained in T#M , then will be called a nontrivial characteristic (resp. nontrivial null characteristic). To each vector eld X onM we associate the function HX : T M ! IR given by HX(q; ) = h ;X(q)i for 2 T qM . Then HX is of class Ck if and only if X is. Moreover, d M( ! HX(x; )) = X(x) for all (x; ) 2 T M . In particular the characteristics of HX are exactly the integral curves of X. The identity fHX ;HY g = H[X;Y ] holds for X;Y 2 V 1(M), and therefore the map X ! HX is a Lie algebra homomorphism from (V 1(M); [ ; ]) to (C1(N); f ; g). If X 2 V 1(M), then the vector eld ! HX is the Hamiltonian lift of X. 3.4 Distributions, admissible curves, orbits Let M be a smooth manifold. A mapping E that assigns to every x 2 M a linear subspace E(x) of TM (resp. of T M) will be called a singular distribution (resp. codistribution) on M . If E is actually a smooth subbundle of TM (resp. of T M) then E will be called a smooth distribution (resp. codistribution) on M . If E is a singular distribution (resp. codistribution), we use E? to denote the annihilator of E, so E? is a singular codistribution (resp. distribution), and E? is smooth if and only if E is. Clearly, (E?)? = E. If E is a smooth distribution on M , an E-admissible curve (or, 20 WENSHENG LIU AND H ECTOR J. SUSSMANN simply, an E-curve) is a locally absolutely continuous curve in M such that _ (t) 2 E( (t)) for almost all t in the domain of . If is an arc |i.e. if the domain of is compact| then is an E-admissible arc or, simply, an E-arc. Remark 3 Admissible curves are sometimes called \horizontal curves" in the literature, because one of the most common ways of constructing interesting smooth distributions E is by letting E be the family of horizontal subspaces of a connection. If p; q 2 M , we write p E q (and say that q is E-reachable from p) if there is an E-arc such that @ = (p; q). We say that M is E-connected if p E q for all p; q 2 M . It is clear that E is an equivalence relation on M . The equivalence classes modulo E are the E-orbits. It can be proved (see, e.g., [24])) that every E-orbit S has a unique topology and di erentiable structure with respect to which S is a smooth connected submanifold of M such that E(x) TxM for all x 2 S. Then the restriction EdS of E to S is a smooth distribution on S, and S is EdS-connected. 3.5 Nonholonomic distributions; regularity If E is a smooth distribution on M , and U M is open, we use Sec(E;U) to denote the set of all smooth sections of E de ned on U , and write Sec(E) = [USec(E;U). For each positive integer k we let Seck(E;U) denote the set of all vector elds X on U such that X is a linear combination with smooth coe cients of iterated brackets of degree k of members of Sec(E;U), and write Seck(E) = [USeck(E;U). For p 2 M , we let Ek(p) denote the set fX(p) : X 2 Seck(E)g. We write kE(p) = dimEk(p) for k 1, and 0E(p) = 0, and let k E(p) = kE(p) k 1 E (p) for k 1. We let E1(p) = [1k=1Ek(p). The rst positive integer k such that kE(p) = dimE1(p) is known as SUB-RIEMANNIAN METRICS 21 the step number of E at p, and we write k = Step(E; p). The sequence of integers ( 1 E(p); 2 E(p); : : : ; k E(p)) is the type of E at p. Any nite sequence ( 1 E(p); 2 E(p); : : : ; j E(p)) with 1 j k will be said to be a partial type of E at p. (For example, E is of partial type (2; 1; 1; 1) at p i the spaces E(p), E2(p), E3(p) and E4(p) are of dimensions 2, 3, 4 and 5, respectively, and E is of type (2; 1; 1; 1) at p if in addition E4(p) = E1(p).) De nition 1 Let k be a positive integer. A smooth distribution E is called k-regular if for every integer j such that 1 j k the number kE(p) is independent of p. If E is k-regular for all k, then we call E regular. For a k-regular distribution E and a j 2 f1; : : : ; kg we use jE (resp. j E) to denote the common value of the numbers jE(p) (resp. j E(p)) and we say that E is of partial type ( 1 E; : : : ; k E). If E is regular, we write Step(E) for the common value of the numbers Step(E; p) and, if k = Step(E), we say that E is of type ( 1 E; : : : ; k E). De nition 2 A smooth distribution E on a manifold M is nonholonomic, or bracket-generating, if E1(p) = TpM for all p 2 M . If E is both regular and nonholonomic, then the number Step(E) is known as the degree of nonholonomy of E. If k is a positive integer, a smooth distribution E such that Ek(p) = TpM for all p 2 M will be called k-generating. De nition 3 A smooth distribution E on M is said to be strongly bracket-generating (SBG) at a point p 2 M if E(p) + [X;Sec(E)](p) = TpM for every X 2 Sec(E) such that X(p) is de ned and 6= 0. Remark 4 The most obvious examples of SBG distributions E are those that are 1-generating, i.e. such that E = TM . SBG distributions other than these are rather exceptional. For example, if m < n and n = dimM , then a necessary condition for a regular m-dimensional 22 WENSHENG LIU AND H ECTOR J. SUSSMANN E on M to be SBG is that m be even, E be 2-generating, and n < 2m. (Examples of other, even more restrictive, necessary conditions are given in [7].) The most commonly encountered situation where E 6= TM but E is SBG is the contact case, i.e. that of a regular bracket-generating distribution of type (2; 1). 4 Abnormal extremals To every smooth distribution E on a smooth manifold M we associate a set HE of Hamiltonian functions T M ! IR by letting HE be the set of all functions HX , as X ranges over all smooth sections of E. De nition 4 Let M be a smooth manifold, and let E be a smooth distribution onM . An abnormal biextremal (resp. abnormal extremal) of E is a locally absolutely continuous curve in T#M (resp. in M) which is a null bicharacteristic (resp. a nontrivial null characteristic) of HE. With the above de nition, the abnormal extremals of E are exactly the projections on M of the abnormal biextremals. In view of the identity d M( ! HX(x; )) = X(x), it is clear that every abnormal extremal of E is an E-curve. It follows from De nition 4 that every abnormal biextremal is contained in the set of common zeros of the family of functions HE, which is precisely the annihilator E? of E. The local structure of E? is described by the following trivial result, stated here for future use. Proposition 3 Let E be an m-dimensional smooth distribution in an n-dimensional manifoldM . Then E? is a smooth (2n m)-dimensional subbundle of the cotangent bundle T M . If V is a set of the form V = ( M) 1(U), where U is the domain of a basis f1; : : : ; fm of sections of E, then E?\V is the subset of V de ned by the equations H1 = : : : = SUB-RIEMANNIAN METRICS 23 Hm = 0, where Hi = Hfi , i.e. Hi(x; ) = h ; fi(x)i. The di erentials dHi( ) are linearly independent at every point 2 V . Moreover, if 2 E? \V , then the vectors ! Hi( ) form a basis of ! HE( ), and d M( ) maps ! HE( ) isomorphically onto E( M( )). Remark 5 If = 2 E?, then ! HE( ) has dimension n+m, and is in fact equal to the set of all v 2 T (T M) such that d M ( )(v) 2 E( M( )). Indeed, write x = M( ), and let v be such that d M ( )(v) 2 E(x). Let f; g be smooth sections of E such that d M ( )(v) = f(x) and a = Hg( ) 6= 0. Let ~ v = ! Hf ( ). Then the vector v ~ v is tangent to the ber T xM , so it can be identi ed with a covector w 2 T xM . Let z = wa . De ne X = f + g, where is a smooth function on M that vanishes at x and is such that d (x) = z. Then ! HX( ) = ~ v +Hg( ) ! ( ). It is easy to see that the vector ! ( ) is tangent to T xM , and equal to z. But then ! HX( ) = v. Remark 6 Proposition 3 and Remark 5 say that ! HE is a singular distribution on T M , whose dimension is equal tom on E? and to n+m on T M E?. (As before, we are letting m = dimE, n = dimM .) If we pull ! HE back to E?, we get an m-dimensional smooth vector bundle ! HEdE? on E?. Since d M( ) maps ! HE( ) isomorphically onto E( M( )) for every 2 E?, we see that the bundle ! HEdE? can be canonically identi ed with the pullback MdE? E of E to E? via the restriction to E? of the projection M . This means, in particular, that every Riemannian metric on E gives rise to a Riemannian metric on ! HEdE?, a fact that will be important later. Remark 7 The abnormal biextremals, as de ned above, are exactly the characteristic curves of E? that lie in T#M . Recall that, if S is a submanifold of a symplectic manifold (N; ), the characteristic singular distribution induced by on S is the map S 3 s! D(s) = K(s)\TsS, 24 WENSHENG LIU AND H ECTOR J. SUSSMANN where K(s) is the orthogonal complement of TsS relative to the 2form , i.e. K(s) = fv 2 TsN : (v;w) = 0 for all w 2 TsSg. A characteristic curve of S is a D-curve in S, i.e. a curve : I ! S such that _ (t) 2 D( (t)) for almost all t 2 I. If S is de ned by equations H1(s) = : : : = Hm(s) = 0, where the m smooth functions have linearly independent di erentials at every point of S, then the tangent space TsS is the set of all vectors v 2 TsN that satisfy hdHi(s); vi = 0 for i = 1; : : : ;m. Equivalently, v 2 TsS if and only if (v; ! Hi(s)) = 0 for i = 1; : : : ;m. So K(s) is the linear span of the vectors ! Hi(s), i = 1; : : : ;m. If H denotes the linear span (over IR) of the functions H1; : : : ;Hm, then a characteristic curve of S is precisely a curve along which the Hi vanish and whose tangent vector _ (t) at almost every point t belongs to !H ( (t)). In our case, if we take N = T M , = M , S = E?, and a curve : I ! T M is contained in a set of the form V = ( M) 1(U), where U is the domain of a basis f1; : : : ; fm of sections of E, then is a characteristic curve of E? if and only if the functions Hi (def =Hfi) vanish along and the vector _ (t) is in the linear span of the ! Hi( (t)) for almost every t. In other words, has to be a null bicharacteristic of the IR-linear span of H1; : : : ;Hm. Now, this linear span is easily seen to be equivalent to HE on V , and then Proposition 2 implies that the null bicharacteristics of HE on V are the same as those of the linear span of the Hi. Remark 8 Let E be a smooth distribution on a manifold M . Let p 2 M . Fix a Riemannian metric G on E. Let DE(p) denote the set of all E-arcs : [0; 1]! M that have nite energy (i.e. are such that R 1 0 k _ (t)k2Gdt < 1) and satisfy (0) = p. Then this set is independent of the choice of the metric G, and Bismut has shown in [3] how to de ne a di erentiable structure on DE(p) such that DE(p) is a Hilbert manifold. There is a well de ned map, namely the endpoint map, Ep : SUB-RIEMANNIAN METRICS 25 DE(p) ! M that assigns to each 2 DE(p) the end point of , i.e. Ep( ) = (1). Then Ep is a smooth map. An E-arc : [0; 1] ! M is called singular if it is a singular point of E (0), i.e. if the di erential of E (0) at is not surjective. The di erential of Ep is easily calculated, and the calculation yields the result that the abnormal extremals are precisely the singular arcs. We conclude this section by singling out two simple cases when the abnormal extremals are trivial. Proposition 4 Let E be a smooth distribution on the smooth manifold M . Then (i) if E = TM then there are no abnormal extremals of E; (ii) if E is strongly bracket-generating but E 6= TM then the abnormal extremals of E are exactly the constant curves. PROOF. Assume : I ! T#M is an abnormal biextremal of E, and write (t) = ( (t); (t)). If Y is any smooth section of E, then HY vanishes along . If E = TM , this implies that all the functions HY , for all smooth vector elds Y on M , vanish along , so (t) = 0 for all t, contradicting the fact that (t) 2 T#M . If E is SBG, let t 2 I be such that _ (t) exists and is equal to ! HX( (t)) for some smooth section X of E. Then fHX ;HY g( (t)) = 0 and HY ( (t)) = 0 for every smooth section Y of E, so (t) annihilates E( (t)) + [X;Sec(E)]( (t)). Since (t) 6= 0, it follows that X( (t)) = 0, i.e. _ (t) = 0. So _ = 0 a.e., and then is constant. 5 Sub-Riemannian manifolds, length minimizers and extremals A Riemannian metric on a bundle E is a smooth section p ! Gp of the bundle E E such that for each p 2 M the bilinear form E(p) E(p) 3 (v;w)! Gp(v;w) 2 IR is symmetric and strictly positive 26 WENSHENG LIU AND H ECTOR J. SUSSMANN de nite. It is clear that one can always construct a Riemannian metric on any subbundle E of TM by just taking a Riemannian metric on TM and restricting it to E. Conversely, every Riemannian metric on a smooth distribution E on M arises in this way, as can be shown by using partitions of unity. De nition 5 A sub-Riemannian structure on a manifold M is a pair (E;G) such that E is a smooth distribution on M and G is a Riemannian metric on E. A sub-Riemannian manifold is a triple (M;E;G) such that M is a manifold and (E;G) is a sub-Riemannian structure on M . 5.1 The sub-Riemannian distance, length minimization and time optimality If (M;E;G) is a sub-Riemannian manifold, and p 2 M , v;w 2 E(p), we use hv;wiG to denote the number Gp(v;w). If p 2M;v 2 E(p), the length kvkG of v is the number hv; vi1=2 G = Gp(v; v)1=2. The length k kG of an E-arc : [a; b] ! M is the integral R b a k _ (t)kGdt. We agree to de ne k kG to be +1 if is not an E-arc. An E-arc is parametrized by arc-length if k _ (t)kG = 1 for almost all t in the domain of . If : [a; b] ! M is E-admissible, then we can de ne another arc ̂, called the arc-length reparametrization of , as follows. We rst de ne a real-valued function on [a; b] by (t) = R t a k _ (s)kGds. Then is a monotonically nondecreasing function on [a; b] with range [0; k kG]. Moreover, if t1 < t2 but (t1) = (t2), then _ (t) = 0 for almost all t 2 [t1; t2], so (t2) = (t1). We can therefore de ne ̂ : [0; k kG] ! M by letting ̂(s) = (t) if (t) = s. Then it can be shown that ̂ is E-admissible, parametrized by arclength, and satis es ̂ = . If E is a smooth distribution on a smooth manifold M , equipped with a Riemannian metric G, then we de ne dG :M M ! IR[f1g by SUB-RIEMANNIAN METRICS 27 dG(p; q) = inffk kG : @ = (p; q)g. The number dG(p; q) is the distance from p to q. Clearly, dG(p; q) < 1 i p E q. If M is connected and E is bracket-generating, then it is well known that dG : M M ! IR is a metric that induces the topology of M . In the general case we can consider, for every E-orbit S of E, the restriction dSG of dG to S S, which turns out to be a metric on S. If the restriction EdS is bracket-generating as a distribution on S, then the metric dSG induces the topology corresponding to the canonical di erentiable structure of S. (The condition that EdS is bracket-generating is always satis ed, for every E-orbit S, if E is a C! distribution on a C! manifold M , but can fail when E is just of class C1. In that case, it is easy to construct examples where the topology induced by dSG is strictly stronger than the one arising from the canonical di erentiable structure of S.) The function dG is known as the sub-Riemannian distance associated to (M;E;G). De nition 6 Let (M;E;G) be a sub-Riemannian manifold. An E-arc : [a; b]!M is called a minimizer if dG( (a); (b)) = k kG. We letM(M;E;G) be the class of all minimizers, andMPAL(M;E;G) be the class of those minimizers that are parametrized by arc-length. We then know that every arc 2 M(M;E;G) is equivalent modulo reparametrization to a ̂ 2 MPAL(M;E;G), so the problem of completely characterizing all minimizers is equivalent to that of characterizing all the minimizers that are parametrized by arc-length. Let (M;E;G) be the class of all E-arcs that satisfy k _ (t)kG 1 for almost all t. It is clear that every E-arc which is parametrized by arclength is in (M;E;G). For an arc 2 (M;E;G), we will use T ( ) to denote the time (or duration) of , i.e. the number T ( ) = b a, where [a; b] = Domain( ). De nition 7 If 2 (M;E;G), we call time-minimizing, or timeoptimal, if T ( ) T (~ ) whenever ~ 2 (M;E;G), @~ = @ . 28 WENSHENG LIU AND H ECTOR J. SUSSMANN De nition 8 If 2 (M;E;G), we call uniquely time-minimizing, or uniquely time-optimal, if T ( ) T (~ ) for every ~ 2 (M;E;G) such that @~ = @ , and equality only holds if ~ = . De nition 9 If : [a; b]!M belongs to (M;E;G), we call locally time-optimal (resp. locally uniquely time-optimal), if there exists a > 0 such that the restriction of to every subinterval of [a; b] of length is time-optimal (resp. uniquely time-optimal). We let T O(M;E;G) be the class of all time-optimal arcs in (M;E;G). We then have MPAL(M;E;G) = T O(M;E;G) : (12) Indeed, if 2 MPAL(M;E;G), then T ( ) = k kG. Suppose there was an arc 2 (M;E;G) such that @ = @ but T ( ) < T ( ). Since k _ (t)kG 1 for almost all t, we have k kG T ( ), and therefore k kG < k kG, contradicting the assumption that was a minimizer. Conversely, suppose that 2 T O(M;E;G). If was not parametrized by arc-length, then we would have k _ (t)kG 1 for almost all t (because 2 (M;E;G)) and k _ (t)kG < 1 for t in a set of positive measure. Therefore k kG < T ( ). If ̂ is the arc-length reparametrization of , then ̂ 2 (M;E;G), @ ̂ = @ , and T ( ̂) = k ̂kG = k kG < T ( ), contradicting the time-optimality of . So is parametrized by arc-length. If was not a minimizer, there would exist a which is parametrized by arc-length and satis es @ = @ and k kG < k kG. But this implies that T ( ) < T ( ), since both and are parametrized by arc-length. Since 2 (M;E;G), is not timeoptimal, and we have reached a contradiction, completing the proof of (12). SUB-RIEMANNIAN METRICS 29 5.2 Extremals Let (M;E;G) be a sub-Riemannian manifold. In view of (12), the problem of characterizing the minimizers for (M;E;G) is equivalent to that of characterizing the time-optimal arcs. It turns out that (M;E;G) is the set of trajectories of a control system, and this fact can be used to obtain a necessary condition for a 2 (M;E;G) to be time-optimal by applying the \Maximum Principle" of Optimal Control Theory. The condition says that has to be an \extremal," so we begin by de ning what it means for a curve to be an extremal. If (p; ) 2 T M , then the restriction dE(p) of to the subspace E(p) of TpM has a well-de ned norm, since E(p) is an inner product space. We will use k kG to denote this norm. De nition 10 The function H : T M ! IR given by H(x; ) = 1 2k k2G (13) is the Hamiltonian of the sub-Riemannian structure (E;G). De nition 11 A normal biextremal of a sub-Riemannian structure (E;G) is a curve in T M such that (i) is a bicharacteristic of the Hamiltonian H of (E;G), and (ii) H does not vanish along . A normal extremal of (E;G) is a curve in M which is the projection of a normal biextremal. De nition 12 A biextremal of a sub-Riemannian structure (E;G) is a curve which is either a normal biextremal of (E;G) or an abnormal biextremal of E. An extremal of (E;G) is a curve which is either a normal extremal of (E;G) or an abnormal extremal of E. With the above de nitions, the normal extremals (resp. abnormal extremals, extremals) are exactly the projections of the normal biextremals (resp. abnormal biextremals, biextremals). Biextremals are always contained in T#M , and extremals in M . 30 WENSHENG LIU AND H ECTOR J. SUSSMANN The Hamiltonian H of (E;G) is a smooth function on T M . Indeed, if U is any open subset ofM on which there exists an orthonormal basis (f1; : : : ; fm) of smooth sections of E, then H(x; ) = 1 2 m X k=1h ; fk(x)i2 for (x; ) 2 ( M) 1(U) ; (14) which in particular proves the smoothness of H. Moreover, if V is an open relatively compact subset of U , and ' :M ! IR is a smooth function such that ' 1 on V and support(') U , then the vector elds gkdef ='fk are global smooth sections of E. If : I ! T#M is an abnormal biextremal of E, then the functions Hgk have to vanish along . Therefore H( (t)) = 0 if (t) 2 ( M) 1(V ), since H = 1 2 PkH2 gk on ( M) 1(V ). Since every point of T M belongs to ( M) 1(V ) for some relatively compact subset V of the domain of a basis of sections of E, we conclude that H vanishes along . Since, by de nition, the value of H along a normal biextremal is 6= 0, we conclude that is not a normal biextremal. Therefore a biextremal is either normal or abnormal, and the two possibilities are mutually exclusive. The analogous statement for extremals is not true, since it is very easy to give examples of curves , 1, 2 such that 1 is a normal biextremal, 2 is an abnormal biextremal, and = o 1 = o 2. This makes it necessary to introduce a further distinction: De nition 13 A strictly abnormal extremal of a sub-Riemannian manifold (M;E;G) is an abnormal extremal of E that is not a normal extremal of (E;G). The smoothness of H clearly implies that the normal biextremals and the normal extremals of a sub-Riemannian structure (E;G) are smooth. The analogous statement for abnormal extremals is false, as can be shown by simple examples (cf. Subsection 9.6 below). SUB-RIEMANNIAN METRICS 31 5.3 The relationship between minimality and extremality The main result relating minimality and extremality is the following theorem, whose proof is given in Appendix B. Theorem 1 Let (M;E;G) be a sub-Riemannian manifold, and let : [a; b] ! M be a length-minimizer parametrized by arc-length. Then is an extremal. Theorem 1 only gives useful information if the class of extremals is not too large, and in the extreme case when all trajectories are extremals it gives no information at all. This case can actually occur: it is easy to show that if S is any orbit of positive codimension, then all E-arcs contained in S are abnormal extremals. In particular, if all E-orbits have positive codimension, then all E-arcs are abnormal extremals and Theorem 1, as stated, gives no information. However, even in that case one can extract useful information out of Theorem 1 by applying it to the sub-Riemannian manifold (S;EdS;GS), where GS is the restriction of G to EdS. (Indeed, it is clear that a minimizer for (M;E;G) which is contained in S is a minimizer for (S;EdS;GS).) This yields a stronger conclusion: Corollary 1 If (M;E;G) is a sub-Riemannian manifold, : [a; b]! M is a length-minimizer parametrized by arc-length, and S is the Eorbit containing , then is an extremal for (S;EdS;GS ), i.e. is the projection of a biextremal t! (t) = ( (t); (t)) of (M;E;G) such that (t) is nonzero as a covector on S, i.e. (t)dT (t)S 6= 0 for all t. Remark 9 Clearly, it is always better to study minimizers by applying Corollary 1 rather than Theorem 1. It turns out that this always yields nontrivial information, because it is easy to prove that, on an Econnected sub-Riemannian manifold (M;E;G), any two points of M 32 WENSHENG LIU AND H ECTOR J. SUSSMANN can be joined by an E-arc which is not an extremal, so not all E-arcs are extremals. A much stronger conclusion holds if E is bracket-generating: the set of E-arcs that are not extremals is generic (cf. [21] for a proof in the real-analytic case) and, even more strongly, the set of extremals is of in nite codimension in the space of E-arcs. In particular, for real-analytic sub-Riemannian manifolds (M;E;G), the situation is as follows: for every restriction (S;EdS;GS) to an E-orbit S, the distribution EdS is bracket-generating, and then the class of extremals that satisfy the stronger conclusion of Corollary 1 is a very small subset of the set of all E-arcs. Many authors include the requirement that E be bracket-generating in the de nition of a sub-Riemannian manifold. We have not done so, but this will mean that many questions about extremals |such as, for instance, the question whether there can exist nonsmooth abnormal extremals| will only be interesting in the bracket-generating case. The following well-known fact gives a partial converse of Theorem 1 for normal extremals: Theorem 2 Let (M;E;G) be a sub-Riemannian manifold. Then every normal extremal is locally optimal. Theorem 2 is a special case of the general fact that trajectories that can be embedded in a smooth eld of extremals are locally optimal, cf. e.g. [15], [4]. An independent derivation was given by Hamenstadt in [12]. (A self-contained version of the classical control-theoretic proof is included, for completeness, in Appendix C.) In view of Theorems 1 and 2, it is natural to ask whether abnormal extremals are also locally optimal. The answer turns out to be negative: in Appendix D we give an example of a real-analytic, C1-rigid abnormal extremal which is not locally optimal, for a bracket-generating subRiemannian structure. SUB-RIEMANNIAN METRICS 33 Finally, one could ask whether minimizers that are not normal exist at all, and how common they are. The rst question was given an a rmative answer by Montgomery in [17]. (A much simpler example has been presented above in Subsection 2.3.) The second one will be answered |for rank two distributions| in the following two sections, where we will de ne a large class of abnormal extremals for which an analogue of Theorem 2 holds. 6 Regular abnormal extremals for ranktwo distributions We now make a more detailed study of the abnormal extremals in subRiemannian manifolds (M;E;G) such that dimE = 2. We rst note that if dimM = 2 then (M;E;G) is in fact a Riemannian manifold, so there are no abnormal extremals. So from now on we will assume that dimM = n 3 and dimE = 2. 6.1 The regular abnormal foliation of a rank-two distribution We begin by studying the abnormal biextremals associated to a twodimensional distribution E. We will show that all these arcs lie in a set AEC(E) |the \abnormal extremal carrier" of E| and that there is a subset RA(E) of AEC(E) |the \regular abnormal set"| such that (i) RA(E) is a submanifold of T M , (ii) RA(E) admits a canonical foliation FE with one-dimensional leaves, (iii) the abnormal biextremals contained in RA(E) are precisely the arcs in RA(E) that are entirely contained in one of the leaves. Moreover, we will show that (under a mild extra condition on E) the set RA(E) is open and dense in AEC(E), thereby justifying the claim that the abnormal biextremals that are contained in RA(E) are the typical abnormal biextremals. 34 WENSHENG LIU AND H ECTOR J. SUSSMANN Recall that E? denotes the annihilator of E. Then E? is a submanifold of T M of codimension two, and it is clear from De nition 4 that every abnormal biextremal has to be contained in E?\T#M . So it may appear natural to de ne a \typical abnormal extremal" to be one that goes through a \generic point of E? \ T#M ." This, however, would only produce a rather trivial and uninteresting class of abnormal biextremals, namely, constant curves in E? \ T#M . (It follows easily from De nition 4 that every constant trajectory t! (p(t); (t)) 2 E?\T#M is an abnormal biextremal.) It turns out that through most points of E? \ T#M there pass no other abnormal biextremals, because along any nonconstant abnormal biextremal a further condition has to hold, namely, the vanishing of the Poisson bracket fHf ;Hgg of the Hamiltonians Hf , Hg of a pair of independent sections of E: Proposition 5 If E is a smooth two-dimensional distribution on a manifold M , then every nonconstant abnormal biextremal of E is contained in (E2)? \ T#M . PROOF. Let be a nonconstant abnormal biextremal. It follows from De nition 4 that is contained in T#M . We now prove that is also contained in (E2)?. It clearly su ces to prove this under the extra assumption that the domain of is a bounded open interval (a; b). Let A be the set of those t 2 (a; b) such that _ (t) exists and belongs to HE( (t)). Then (a; b) A has measure zero. Let I denote the set of those t 2 (a; b) such that is constant on the interval (t "; t+ ") for some " > 0. Then I is an open subset of (a; b), and I 6= (a; b). Let t 2 (a; b) and let (t) = (p; ). Let U be a neighborhood of p inM such that there are smooth global sections f ,g of E that are independent at every point of U . Let V = ( M) 1(U). Let " > 0 be such that (s) 2 V for s 2 (t "; t + "). If X is any section of E, then there are smooth functions , on U such that X = f + g on U . Then ! HX = ! Hf + ! Hg+Hf ! +Hg ! on V . In particular, sinceHf andHg vanish along , SUB-RIEMANNIAN METRICS 35 we have ! HX( (s)) = ( ! Hf + ! Hg)( (s)) whenever (s) 2 V . For every s 2 (t "; t+ ")\A, we have _ (s) 2 ! HE( (s)), which means that there exists a smooth section Xs of E such that _ (s) = ! HXs( (s)). But then ! HXs( (s)) is a linear combination of ! Hf ( (s)) and ! Hg( (s)). Write ! HXs( (s)) = '(s) ! Hf ( (s))+ (s) ! Hg( (s)). Since Hf vanishes along , the Poisson bracket fHXs;Hfg must vanish at (s). If (s) 6= 0, this implies that fHg;Hfg( (s)) = 0. If '(s) 6= 0, then the same conclusion follows using the fact that Hg vanishes along . So fHg;Hfg( (s)) = 0 for every s 2 A \ (t "; t+ ") such that _ (s) 6= 0. Now assume that t = 2 I. Then there cannot exist a > 0 such that _ (s) = 0 for all s 2 A\(t ; t+ ). So every interval (t ; t+ ) contains an s 2 A for which _ (s) 6= 0, and then fHg;Hfg( (s)) = 0. Since and fHg;Hfg are continuous, we conclude that fHg;Hfg( (t)) = 0. Since fHg;Hfg = H[f;g], we have H[f;g]( (t)) = 0. So h ; f(p)i = h ; g(p)i = h ; [f; g](p)i = 0. Since f(p), g(p) and [f; g](p) linearly span E2(p), we have shown that (p; ) 2 (E2)?. So far, we have established that (t) 2 (E2)? for all t 2 (a; b) such that t = 2 I. Suppose now that t 2 I. Let J be the connected component of I that contains t. Then J is an open subinterval of (a; b), and J 6= (a; b). So there is a point in the closure of J relative to (a; b) such that = 2 J , and therefore = 2 I. Since is constant on J , we have (t) = ( ). But, since = 2 I, we already know that ( ) 2 (E2)?. So (t) 2 (E2)?. Remark 10 As pointed out before, every constant trajectory in E? is an abnormal biextremal. Naturally, such an abnormal biextremal need not be contained in (E2)?. Having shown that all the nonconstant abnormal biextremals must be contained in (E2)? \ T#M , it is natural to call the set AEC(E)def =(E2)? \ T#M 36 WENSHENG LIU AND H ECTOR J. SUSSMANN the abnormal extremal carrier of E, provided that we show that all, or almost all, the points in AEC(E) belong to nonconstant abnormal biextremals. More precisely, we will de ne a subset RA(E) of AEC(E) which, under a mild restriction on E, turns out to be open and dense in AEC(E). This subset will be a submanifold of E? of codimension one, and will admit a one-dimensional foliation FE |de ned by a line subbundle LE of the tangent bundle T RA(E)| with the property that every arc contained in a leaf of FE is an abnormal biextremal of E. To de ne RA(E), LE and FE we must rst study the local structure of the set (E2)?. Pick an open subset U of M on which there exists a basis (f; g) of smooth sections of E, and write V = ( M ) 1(U). Then (E2)?\V is the subset of E?\V de ned by the equation fHf ;Hgg = 0. On the other hand, the tangent space to E? at a point 2 E?\V is the set of all vectors v 2 T T M that satisfy hdHf ( ); vi = hdHg( ); vi = 0. So T E? is precisely the orthogonal subspace |relative to the form M| of the two-dimensional subspace ! HE( ). The restriction of M( ) to ! HE( ) is nonsingular if fHf ;Hgg( ) 6= 0, and vanishes identically if fHf ;Hgg( ) = 0. So 2 (E2)? if and only if ! HE( ) T E?. De nition 14 The set RA(E)def =(E2)? (E3)? is the regular abnormal set of E. A point (p; ) 2 T M is a regular abnormal point (RAP) for E if 2 E2(p)? E3(p)?, i.e. if (p; ) 2 RA(E). Suppose = (p; ) 2 RA(E), and let U , V , f , g be as before. The subset (E2)? of the submanifold E? is de ned, near , by the equation fHf ;Hgg = 0. Since 2 (E2)?, the space ! HE( ) is a subset of T E?. So the vectors ! Hf ( ) and ! Hg( ) are tangent to E? at . On the other hand, since = 2 (E3)?, at least one of the numbers h ; [f; [f; g]](p)i, h ; [g; [f; g]](p)i is 6= 0. This says that the directional derivative of fHf ;Hgg in at least one of the directions ! Hf ( ), ! Hg( ) is 6= 0. Therefore the di erential at of the restriction to E? of SUB-RIEMANNIAN METRICS 37 fHf ;Hgg is 6= 0. This implies that, in a neighborhood of , the set S = f 2 E? : fHf ;Hgg( ) = 0g is a submanifold of E? of codimension one. Moreover, it also implies that the space ! HE( ) |which, as we know, is a subset of the tangent space T E?| is not entirely contained in the tangent space T S, so ! HE( ) \ T S is one-dimensional. Finally, it is clear that S is, near , the same as (E2)?, and also the same as (E2)? (E3)?, since 2 (E2)? (E3)? and (E3)? is closed. We have therefore proved that RA(E)def =(E2)? (E3)? is a submanifold of E? of codimension one, and that the two-dimensional space ! HE( ) is tangent to E? but not to RA(E) at every point of RA(E). This makes it possible to de ne a line bundle LE on RA(E) by letting LE( ) = ! HE( ) \ T RA(E) ; (15) so LE( ) is a 1-dimensional subspace of T RA(E). Then LE is a smooth line subbundle of the tangent bundle of RA(E). It is clear that LE( ) ! HE( ) for every 2 RA(E). So every absolutely continuous curve : [a; b] ! RA(E) that satis es _ (t) 2 LE( (t)) for almost all t is a bicharacteristic of HE, and is obviously null, since RA(E) E?. Therefore every such is an abnormal biextremal of E. Conversely, every abnormal biextremal of E which goes through a point 2 RA(E) at a time t must satisfy (s) 2 RA(E) for all s 2 (t "; t+ "), and _ (s) 2 LE( (s)) for almost all s 2 (t "; t + "), if " > 0 is small enough. (This is obviously true if is constant. If is nonconstant, Proposition 5 implies rst of all that is contained in (E2)?, and then (s) must belong to RA(E) for s near t, since RA(E) is relatively open in (E2)?. But then, for almost all s, _ (s) must belong to ! HE( (s)) and be tangent to RA(E), so _ (s) 2 LE( (s)).) Clearly, LE is an involutive distribution on E. We use FE to denote the set of leaves of LE, and refer to FE as the regular abnormal foliation induced by E. We summarize the above into the following 38 WENSHENG LIU AND H ECTOR J. SUSSMANN Proposition 6 If E is a smooth two-dimensional distribution on a manifold M such that dimM 3, then the set RA(E)def =(E2)? (E3)? of regular abnormal points of E is a smooth submanifold of codimension three of T#M , and a smooth submanifold of codimension one of E?. On RA(E) there is a one-dimensional subbundle LE of T RA(E) (de ned by letting LE( )def = ! HE( ) \ T RA(E)) that gives rise to a foliation FE of RA(E), with the property that an arc in RA(E) is an abnormal biextremal of E if and only if it is contained in a leaf of FE. It will be useful to have an explicit description of a section of LE. Let = (p; ) 2 RA(E). Let (f; g) be, as before, a basis of sections of E near p. Then a linear combination v = a ! Hf ( ) + b ! Hg( ) is in LE( ) if and only if vfHf ;Hgg = 0, that is, if and only if afHf ; fHf ;Hggg( ) + bfHg; fHf ;Hggg( ) = 0 : The fact that 2 RA(E) implies that fHf ; fHf ;Hggg( )2 + fHg; fHf ;Hggg( )2 6= 0 So we can get a nonzero element Vf;g( ) 2 LE( ) by choosing a = fHg; fHf ;Hggg( ) ; b = fHf ; fHf ;Hggg( ) : ThenVf;g( ) = fHg; fHf ;Hggg( ) ! Hf ( ) fHf ; fHf ;Hggg( ) ; (16) and we have proved: Proposition 7 If = (p; ) 2 RA(E) and (f; g) is a basis of sections of E in a neighborhood of p, then LE( ) is spanned by the vector Vf;g( ) given by (16). An important corollary of Proposition 7 is the following observation: SUB-RIEMANNIAN METRICS 39 Corollary 2 If E is a smooth two-dimensional distribution on a manifold M , then the line subbundle LE of the tangent bundle of RA(E) has a canonical orientation. PROOF. We de ne the orientation to be the one that makes the vector Vf;g( ) positive, where (f; g) is an arbitrary basis of sections near p. Notice that, if two bases (f; g), (F;G) of sections are positively related, then we can deform (F;G) continuously to (f; g) near p, and produce a smooth one-parameter family of bases (fs; gs) such that (f0; g0) = (f; g) and (f1; g1) = (F;G). Since Vfs;gs( ) never vanishes, we see that VF;G( ) and Vf;g( ) are positive multiples of each other. On the other hand, direct inspection of the formula for Vf;g( ) shows that Vf;g( ) = Vg;f ( ). So the orientation determined by Vf;g( ) does not change when we reverse the orientation of the basis (f; g) either. We now turn to the question of the genericity of the regular abnormal points within the abnormal extremal carrier AEC(E). We show that the RAP's form a relatively open dense subset of AEC(E), provided that E satis es a mild condition that holds, in particular, for generic distributions and for all regular bracket-generating distributions. To state the condition, we rst de ne, for each nite sequence (d1; : : : ; dm) of positive integers, and each smooth distribution E on a manifold M , a set SE(d1; : : : ; dm) by letting SE(d1; : : : ; dm) = fp 2M : dimEk(p) = dk for k = 1; : : : ;mg : (17) De nition 15 A two-dimensional distribution E on a manifold M is weakly regular if SE(2; 2; 2) Clos SE(2; 2) SE(2; 2; 2) : (18) Proposition 8 If E is a two-dimensional weakly regular smooth distribution on a manifold M , then RA(E) is a relatively open dense subset of (E2)? \ T#M . 40 WENSHENG LIU AND H ECTOR J. SUSSMANN PROOF. It is clear that RA(E) is relatively open in (E2)?. To prove that it is dense in (E2)? \ T#M , we pick a (p; ) 2 (E2)? \ T#M , and show that (p; ) is a limit of a sequence f(pj; j)g of points in (E2)? (E3)?. We consider rst the case when dimE3(p) 3. If E3(p) 6= E2(p), we can just take all the pj equal to p, and approximate by a sequence f jg of functionals in E2(p)? E3(p)?. If E3(p) = E2(p), then necessarily dimE2(p) = 3, and dimM > 3 because 6= 0. Then dimE2(x) = 3 for x in a neighborhood of p. On the other hand, if dimE3(x) was equal to 3 for x in some neighborhood U of p, it would follow that E2 is involutive near p, so that E1(x) = E2(x) 6= TxM for x near p, contradicting the bracket-generating property of E. So there exist points pj 2 U , converging to p, such that dimE3(pj) > 3. Since dimE2(pj) = 3, there exist ~ j 2 T pjM such that ~ j ! and ~ j 2 E2(pj)?. Then we can approximate ~ j by j 2 E2(pj)? E3(pj)?, obtaining a sequence f(pj ; j)g of RAP's that converges to (p; ). We now turn to the remaining case, namely, when dimE3(p) = 2. Then p 2 SE(2; 2; 2), and our hypothesis says that p is the limit of a sequence fpjg such that pj 2 SE(2; 2) but pj = 2 SE(2; 2; 2). Then = lim ~ j , ~ j 2 E2(pj)?, and ~ j can be approximated by j 2 E2(pj)? E3(pj)?, since E3(pj) 6= E2(pj). So, once again, (p; ) is the limit of a sequence of points in RA(E). Finally, we show that weak regularity is a generic property: Proposition 9 On manifolds of dimension 3: (i) all regular twodimensional distributions are weakly regular, (ii) weak regularity is a generic property of two-dimensional distributions. PROOF. In the regular case, SE(2; 2; 2) is empty. For generic twodimensional distributions on an n-dimensional manifold, SE(2; 2; 2) is a submanifold of codimension = 1 2n(n 1)(n 2). If n 4, then > n, so SE(2; 2; 2) is empty. If n = 3, then = 3, so SE(2; 2; 2) is a SUB-RIEMANNIAN METRICS 41 discrete set. However, SE(2; 2) is a smooth two-dimensional surface, so every p 2 SE(2; 2; 2) is a limit of points in SE(2; 2) that do not belong to SE(2; 2; 2), so (18) is true in this case as well. 6.2 Regular abnormal biextremals of a subRiemannian manifold We now assume that (M;E;G) is a sub-Riemannian manifold such that dimE = 2 and dimM 3. In that case, as explained in Remark 6, the metricG on E induces a metric on the bundle ! HEdE?, since this bundle is canonically identi ed with the pullback of E to E? via MdE?. This metric can be restricted to ! HEdRA(E), and then to the subbundle LE of ! HEdRA(E). Moreover, using the canonical orientation of LE described in Corollary 2, we can de ne a vector eld XE, by choosing XE( ), for each 2 RA(E), to be the vector in LE( ) that has length one and is positively oriented. If (f; g) is an orthonormal basis of sections of E near M( ), then XE( ) = fHg; fHf ;Hggg( ) ! Hf ( ) fHf ; fHf ;Hggg( ) ! Hg( ) qfHf ; fHf ;Hggg( )2 + fHg; fHf ;Hggg( )2 : (19) A smooth curve in RA(E) which is an integral curve of XE or of XE will be called a regular abnormal biextremal of the sub-Riemannian manifold (M;E;G). A curve : [a; b] ! M which is a projection of a regular abnormal biextremal will be called a regular abnormal extremal of (M;E;G). Naturally, through every point of RA(E) there pass exactly two maximal regular abnormal biextremals, namely, the maximal integral curves of XE and of XE. If U is the domain of a coordinate chart and of an orthonormal basis (f; g) of sections of E, then the equation of the integral trajectories of XE that are contained in ( M) 1(U) \ RA(E) is _ x = h ; [g; [f; g]](x)if(x) h ; [f; [f; g]](x)ig(x) qh ; [g; [f; g]](x)i2 + h ; [f; [f; g]](x)i2 ; (20) 42 WENSHENG LIU AND H ECTOR J. SUSSMANN _ = h ; [g; [f; g]](x)i @f @x(x) + h ; [f; [f; g]](x)i @g @x(x) qh ; [g; [f; g]](x)i2 + h ; [f; [f; g]](x)i2 ; (21) but it is worth emphasizing that the ow on RA(E) de ned by these equations is intrinsic, and depends only on the distribution E and the metric G. We also point out that an arc t ! (x(t); (t)) is a solution of (20) and (21), i.e. an integral curve of XE, if and only if t ! (x(t); (t)) is an integral curve of XE. Therefore, if is a regular abnormal extremal that happens to be a projection of an integral curve of XE, then we can realize as a projection of an integral curve of XE by just reversing the sign of the adjoint vector . So we have shown: Corollary 3 If (M;E;G) is a sub-Riemannian manifold such that dimE = 2, then a curve in M is a regular abnormal extremal if and only if it is a projection of an integral curve of XE. In particular, through every point p 2 M such that E3(p) 6= E2(p) there passes a regular abnormal extremal. In our study of optimal abnormal extremals arising from abnormal biextremals in RA(E), we will restrict our attention to regular abnormal extremals. This is possible because, to begin with, every E-arc is equivalent up to time reparametrization to a trajectory parametrized by arc-length, so we can limit ourselves to considering abnormal extremals parametrized by arc-length. If is such an extremal, and = M o , where : [a; b]! RA(E) is an abnormal biextremal, then is an LEarc in RA(E), also parametrized by arc-length. Moreover, if we de ne a curve : I ! M to be simple if it has no double points (i.e. if is a one-to-one map), and locally simple if I can be covered by relatively open subintervals on which is simple, then it is clear that an arc : [a; b] ! M cannot minimize length unless it is simple. A fortiori, if minimizes length and = M o , then has to be locally simple. The following trivial remark then implies that has to be a regular abnormal biextremal. SUB-RIEMANNIAN METRICS 43 Proposition 10 Let S be a smooth manifold, and let L be a smooth 1-dimensional subbundle of TS, endowed with a Riemannian metric. Let : I ! M be an L-curve parametrized by arc-length. Then the following three conditions are equivalent: (i) is smooth, (ii) is locally simple, (iii) is locally an integral curve of a smooth section of L of length one. Corollary 4 If : [a; b]! RA(E) is an abnormal biextremal arc such that = M o minimizes length and is parametrized by arc-length, then is a regular abnormal biextremal. 7 Local optimality of regular abnormal extremals Our strategy for proving that regular abnormal extremals are locally optimal will consist of showing that, locally, the equations of a regular abnormal extremal can be put in a \normal form" that is particularly suited for proving optimality. We rst introduce a new de nition. De nition 16 Let (M;E;G) be a sub-Riemannian manifold such that dimE = 2. An E-arc : [a; b] ! M , parametrized by arc-length, is a Nice Abnormal Extremal (NAE) if there exist U , Y , ! such that the following conditions are satis ed: (1) U is an open subset of M such that ([a; b]) U ; (2) Y is a smooth section of E with domain U such that kY (p)kG = 1 for all p 2 U ; (3) is an integral curve of Y , i.e. _ (t) = Y ( (t)) for almost all t 2 [a; b] (that is, is of class C1 and _ (t) = Y ( (t)) for all t 2 [a; b]); 44 WENSHENG LIU AND H ECTOR J. SUSSMANN (4) ! is a smooth 1-form on U , such that (i) !( (t)) annihilates E( (t)) for all t 2 [a; b], (ii) for every p 2 U , the linear functional !(p) does not belong to the annihilator of E3(p), and (iii) the Lie derivative LY ! of ! with respect to Y vanishes identically along . The following theorem describes the precise relationship between regular and nice abnormal extremals. Theorem 3 Every NAE is a regular abnormal extremal. Conversely, if : [a; b] ! M is a regular abnormal extremal then is locally a NAE, in the sense that for every t 2 [a; b] there exists an " > 0 such that the restriction of to [t "; t+ "] \ [a; b] is a NAE. PROOF. Let : [a; b] ! M be a NAE, and let U , Y , ! be such that (1), (2), (3), (4) hold. Let (t) = !( (t)), so is a eld of covectors along . Let (t) = ( (t); (t)), so is a smooth curve in T M . Since (t) 2 E( (t))? for all t 2 [a; b], the curve is in fact contained in E?. If Z is any smooth vector eld on U , then Y h!;Zi = hLY !;Zi+h!; [Y;Z]i. Therefore Y h!;Zi = h!; [Y;Z]i along . In particular, if Z is a section of E, then h!;Zi vanishes along , so Y h!;Zi 0 along , and then h!; [Y;Z]i vanishes along as well. This implies that (t) 2 E2( (t))? for each t 2 [a; b]. So is in fact contained in (E2)?, and Condition (4.ii) implies that is contained in RA(E). If Z is an arbitrary smooth vector eld on U , then the fact that the equation Y h!;Zi = h!; [Y;Z]i holds along says that d dth (t); Z( (t))i = h (t); [Y;Z]( (t))i ; (22) that is, d dtHZ( (t)) = H[Y;Z]( (t)) = fHY ;HZg( (t)) = ! HYHZ( (t)) : (23) SUB-RIEMANNIAN METRICS 45 In other words, the directional derivative of HZ at (t) in the direction of _ (t) is equal to the derivative in the direction of ! HY ( (t)), for every Z. It is easy to see that, if 2 T#M , then the set fdHZ( ) : Z 2 V 1(U)g is the whole cotangent space T  (T M). (In coordinates, we have ! HZ(x; ) = (Z(x); @Z @x (x)). If 6= 0, then we can clearly choose Z such that Z(x) = v and @Z @x (x) = w for any given v, w.) Hence h ; _ (t)i = h ; ! HY ( (t)i for every 2 T (t)(T M). So _ (t) = ! HY ( (t)). Therefore _ (t) 2 LE( (t)). This proves that is a regular abnormal biextremal. We now prove the converse. Let : [a; b] ! M be a regular abnormal extremal, and let : [a; b] ! RA(E) be a regular abnormal biextremal such that M o = . Write (t) = ( (t); (t)). Since the result we are trying to prove is local, and is smooth and such that k _ (t)kG = 1 for all t, we may assume that M is an open subset of IRn, E admits a global basis (f; g) of sections which is orthonormal with respect to the metric G, and _ (t) = ( f+ g)( (t)), where and are smooth functions on M that satisfy 2 + 2 1. We then de ne the vector eld Y on M by Y = f + g. Then Y is a smooth section of E on M such that kY (x)kG = 1 for all x 2 M . We also de ne a smooth vector eld W on T M by W (x; ) = (x) ! Hf (x; ) + (x) ! Hg(x; ) : Since is a regular abnormal biextremal, we have _ (t) = u(t) ! Hf( (t)) + v(t) ! Hg( (t)) and _ (t) = d M ( (t)) _ (t) = u(t)f( (t)) + v(t)g( (t)) so that u(t) = ( (t)) and v(t) = ( (t)). Therefore _ (t) = ( (t)) ! Hf ( (t)) + ( (t)) ! Hg( (t)) = W ( (t)) : 46 WENSHENG LIU AND H ECTOR J. SUSSMANN So is an integral curve of W . We now pick a t 2 [a; b], write p = ( t), and let S be a hypersurface through p such that Y ( p) = 2 T pS. We let ! be a smooth section of T M on S such that !( p) = ( t). For each p 2 S, let p be the integral curve of W that goes through (p; !(p)) at time t. Write p(t) = ( p(t); p(t)). By shrinking S, if necessary, we may assume that there is an " > 0 such that all the curves p, for p 2 S, are de ned on the interval ( t "; t+"), and the map (p; t)! p(t) is a di eomorphism from S ( t "; t+") onto an open subset U of M . We can then de ne a 1-form ! on U by letting !( p(t)) = p(t). It is clear that p on J = [a; b] \ ( t "; t+ "). So !( (t)) = (t) for t 2 J . In particular, !( (t)) 2 E( (t))? for all t 2 J , and !( p) = 2 E3( p)?. So, after shrinking S and ", if necessary, we can assume that !(p) = 2 E3(p)? for all p 2 U . To conclude our proof, we have to show that LY !( (t)) = 0 for all t 2 J . Now, if Z is a smooth vector eld on U , then hLY !;Zi = Y h!;Zi h!; [Y;Z]i : (24) So our conclusion will clearly follow if we prove that the derivative of h (t); Z( (t))i is h (t); [Y;Z]( (t))i. This is equivalent to showing that the derivative of HZ along is H[Y;Z], i.e. fHY ;HZg. This will follow if we prove that _ (t) = ! HY ( (t)). Since _ (t) = W ( (t)), it su ces to prove that ! HY and W agree on . Now W = ! Hf + ! Hg, while Y = f + g, so that HY = Hf + Hg, and then ! HY = ! Hf + ! Hg+ Hf ! + Hg ! . Therefore ! HY = W + Hf ! + Hg ! . Since Hf and Hg vanish along , we conclude that ! HY W on , and the proof is complete. 7.1 The main inequality The following lemma was originally announced in the preprint [25]. (The concept of a \control system" is de ned in Appendix B.) SUB-RIEMANNIAN METRICS 47 Lemma 2 Let n 3, and let U be an open subset of IRn. Let ', 1; 2; 4; : : : ; n, 1; 3; : : : ; n be smooth functions on U , and let be the control system _ x1 = u'(x) + vx1 1(x) ; _ x2 = (1 + x1 2(x))v ; _ xj = vx1 j(x) for j = 3; : : : ; n ; where 3 = x1 1 + x3 3 + : : :+ xn n ; (25) and the controls u, v are subject to the constraint u2 + v2 1. Let x : [a; b]! U be a trajectory of , corresponding to the control functions u(t) 0, v(t) 1, and starting at a point x = x (a) = (0; x2; 0; : : : ; 0) (so that x (t) = (0; x2 + t a; 0; : : : ; 0) for a t b). Assume that 1(x (t)) 6= 0 for a t b. Then x is locally uniquely time-optimal for . Remark 11 The important point in Formula (25) is that the righthand side does not contain a term of the form x2 2. This fact will play a crucial role in the proof of Lemma 2. Consequently, in our proof of the local optimality of regular abnormal extremals, the crucial step will involve changing coordinates to put our equations in the form of Lemma 2, with 3 of the form (25), for which it will be necessary to prove that the expression for 3 does not contain a term of the form x2 2 . PROOF. Let K = fx (t) : a t bg, so K is a compact subset of U . Let V U be open, and such that K V and the closure V of V is compact and contained in U . Then there exists a constant C1 > 0 such that k _ x(t)k C1 whenever x( ) is a trajectory of which is contained in V . Let = min fkx yk : x 2 K; y 2 IRn V g : (26) 48 WENSHENG LIU AND H ECTOR J. SUSSMANN Then > 0. Let ̂1 be such that 0 < ̂1 < C1 . Then, if x( ) : [t1; t2]! U is a trajectory of that goes through a point of K and is such that t2 t1 ̂1, it follows that x( ) is entirely contained in V . We let C2 = sup fj i(x)j : x 2 V ; i = 1; : : : ; ng ; C3 = sup( j i(x)j j 1(x (t))j : a t b ; x 2 V; i = 1; 3; : : : n) ; = inf fj 1(x (t))j : a t bg : Let ̂2 > 0 be such that, whenever x; y 2 V and kx yk C1̂2, then it follows that j 1(x) 1(y)j 4 . Let ̂3 = 4(n 2)C2C3 1 ; (27) and pick a ̂4 > 0 such that ̂4 < (3C1C2) 1. Let ̂ = min(̂1; ̂2; ̂3; ̂4). Now let a t1 t2 b be such that = t2 t1 ̂ . Let be the restriction of x to the interval [t1; t2]. We will show that is uniquely time-optimal. Assume that : [s1; s2] ! U is another trajectory of that goes from (t1) to (t2) in time = s2 s1 and corresponds to control functions u ; v de ned on [s1; s2] such that u (s)2 + v (s)2 1 for all s 2 [s1; s2]. We will show that , with equality holding if and only if (s) = (s s1 + t1) for s1 s s2. Let (s) = ( 1(s); 2(s); 3(s); (s)), where : [s1; s2] ! IRn 3. Let us assume that . Since ̂ , and goes through a point of K, it follows (since ̂ ̂1) that is entirely contained in V , and the bound k _ (s)k C1 holds for almost all s. In particular, we have j _ 1(s)j C1 for almost all s. Let h(s) = R s s1 v (r)dr. Then jh(s)j s s1 for all s, so h(s2) . Let = h(s2), = sup fj 1(s)j : s1 s s2g. We then have = x 2(t2) x 2(t1) = 2(s2) 2(s1) = Z s2 s1 1 + 1(s) 2( (s)) v (s)ds SUB-RIEMANNIAN METRICS 49 Z s2 s1 v (s)ds+ C2 = h(s2) + C2 = + C2 : On the other hand, Z s2 s1 1(s)2ds = Z s2 s1 1(s)2(1 v (s))ds + Z s2 s1 1(s)2v (s)ds : (28) The rst integral of the right-hand side is bounded by 2 . The second integral is equal to Z s2 s1 1(s)2v (s) 1( (s1)) 1( (s1))ds ; i.e. to Z s2 s1 1(s)2v (s) 1( (s1)) 1( (s)) 1( (s1)) ds+ Z s2 s1 1(s)2v (s) 1( (s)) 1( (s1))ds : But 0 = 3(s2) 3(s1) 1( (s1)) = Z s2 s1 1(s)2v (s) 1( (s)) 1( (s1))ds+ n Xi=3 Z s2 s1 1(s) i(s)v (s) i( (s)) 1( (s1))ds : So Z s2 s1 1(s)2ds 2 + Z s2 s1 1(s)2v (s) 1( (s1)) 1( (s)) 1( (s1)) ds n Xi=3 Z s2 s1 1(s) i(s)v (s) i( (s)) 1( (s1))ds : For i 3, the functions i satisfy _ i = v 1 i( ). Therefore, since i(s1) = 0, we have j i(s)j C2 1 2 Z s2 s1 1(s)2ds 12 : (29) Then Z s2 s1 1(s) i(s)v (s) i( (s)) 1( (s1))ds C2C3 Z s2 s1 1(s)2ds : (30) 50 WENSHENG LIU AND H ECTOR J. SUSSMANN Also, Z s2 s1 1(s)2v (s) 1( (s1)) 1( (s)) 1( (s1)) ds 14 Z s2 s1 1(s)2ds ; (31) since j 1( (s1))j and j 1( (s1)) 1( (s))j 4 , because k (s1) (s)k C1̂2 (since s s1 ̂2 and k _ k C1). So we get the boundZ s2 s1 1(s)2ds 2 + 1 4 + (n 2)C2C3 Z s2 s1 1(s)2ds : (32) Since ̂3 4(n 2)C2C3 1, we have the bound R s2 s1 1(s)2ds 2 + 1 2 R s2 s1 1(s)2ds, from which it follows that R s2 s1 1(s)2ds 2 2 . Now let ~ s 2 [s1; s2] be such that j 1(~ s)j = . Since 1(s1) = x 1(t1) = 0, 1(s2) = x 1(t2) = 0, and j _ 1j C1, we have ~ s s1 C1 and s2 ~ s C1 . Hence the intervals I1 = [~ s C1 ; ~ s] and I2 = [~ s; ~ s + C1 ] are entirely contained in [s1; s2]. On each of these intervals Ij, j 1(s)j is bounded below by the linear function j which is equal to at ~ s and to zero at the other endpoint. Clearly, the integral of 2j over Ij is exactly 3 3C1 . So R s2 s1 1(s)2ds RI1 1(s)2ds + RI2 1(s)2ds 2 3 3C1 . Combining the upper and lower bounds for R s2 s1 1(s)2ds we get 2 3 3C1 2 2 . Therefore 3C1 . But then + C2 + 3C1C2 = + 3C1C2 1 : (33) Since ̂4 < (3C1C2) 1, we have 3C1C2 1 0. Therefore . So we have shown that implies . Therefore cannot be < . So is optimal, as stated. Moreover, the inequality is in fact strict unless = 0 (since 3C1C2 < 1). So = can only happen if = 0, i.e. if v = 1 almost everywhere. But then u = 0 almost everywhere, and (s) = (s s1 + t1). So is uniquely optimal. SUB-RIEMANNIAN METRICS 51 7.2 The normal form theorem We now show that a nice abnormal extremal can be put in the form given by Lemma 2. First, we need to introduce some de nitions and notations, and prove a lemma on invariant distributions. De nition 17 Let M be a manifold, and let D be a smooth distribution on M . Let X be a smooth section of D. If : [a; b] ! M is a curve in M , we say that D is invariant under X along if for every vector eld Y 2 Sec(D) the vector [X;Y ]( (t)) belongs to D( (t)) for all t 2 [a; b]. Whenever = (x1; : : : ; xn) is a cubic chart with domain U , we will automatically identify the domain U with the cube Cn( ), so all smooth functions and vector elds on U will be viewed as functions or vector elds on Cn( ). Lemma 3 Let M be a manifold of dimension n 3. Let D be a smooth distribution of dimension k < n on M . Let p 2 M , and let Y be a smooth section of D. Assume that > 0, : [ ; ] ! M is an integral curve of Y passing through p at time 0, and D is invariant under Y along . Then for every X 2 Sec(D) such that X(p); Y (p) are linearly independent there exist a 2 (0; ) and a cubic chart = (x1; : : : ; xn) of radius , centered at p, with respect to which X = @ @x1 and Y = @ @x2 + x1 n X k=1 k(x) @ @xk ; (34) where the k are smooth real-valued functions on Cn( ), and 3 is of the form (25). Moreover, if [X; [X;Y ]](p) 62 D(p), then can be taken such that @ 3 @x1 (0) 6= 0. Remark 12 If , are as in the conclusion of Lemma 3, then it is clear that the curve ~ : t ! (0; t; 0; : : : ; 0), t 2 ( ; ), is an integral trajectory of Y . So ~ is the restriction of to ( ; ). In other words: 52 WENSHENG LIU AND H ECTOR J. SUSSMANN relative to the chart , the restriction of to ( ; ) is contained in the domain of and is given by (t) = (0; t; 0; : : : ; 0). Before we prove Lemma 3, we brie y review the exponential formalism for ows. If X is a vector eld, we use exponential notation for the ow of X, and have the maps etX act on the right, so t! petX denotes the integral curve of X that passes through p at time t = 0. It is also convenient to write the vector elds as acting on points on the right, i.e. to use pX as an alternative notation for X(p). (The reader may just view this as no more than a notational convention. The deep reason for this convention is that vector elds are rst-order di erential operators on functions, and act on functions on the left. Points are dual objects to functions |Dirac Delta distributions, to be precise| and therefore vector elds should act on points on the right. In addition, having the vector elds and their exponentials act on the right has the extra advantage that Lie-theoretic formulae such as Campbell-Hausdor come out right if one chooses to de ne the Lie bracket via [X;Y ]def =XY Y X, whereas, if the left action notation were used, consistency would require that the bracket [X;Y ] be de ned as Y X XY .) If : M ! N is a smooth map, and v is a tangent vector to M at p, then v will denote the tangent vector at (p) which is usually denoted by v, or d (p)(v). With these notations, for any two smooth vector elds X;Y on a smooth manifold M , and any t such that e tY is de ned, pe tYXetY is a tangent vector at p, and the following formula is true: d dt pe tYXetY = pe tY [X;Y ]etY : (35) Lemma 4 Let Y be a smooth vector eld on a manifold M . Let p 2 M be a point and let J be an interval such that 0 2 J and petY is de ned for all t 2 J . Let : J ! M be de ned by (t) = petY . Let D be a smooth k-dimensional distribution on M which is invariant under Y along . Then, for any pair of points qi = petiY , ti 2 J , i = 1; 2, it SUB-RIEMANNIAN METRICS 53 follows that ve(t1 t2)Y 2 D(q1) whenever v 2 D(q2) : (36) PROOF. First, pick an arbitrary point p = ( t) on , and choose a basis (X1; : : : ;Xk) of sections of D on a neighborhood U of p. Then, since D is invariant under Y along , we can write ( pe tY )[Xi; Y ] =Xj ij(t)( pe tY )Xj for each t such that pe tY 2 U , where the ij are smooth functions. So, if we let vi(t) = pe tYXietY , and let t vary on an interval I = ( "; "), chosen so that pe tY 2 U for t 2 I, we can conclude that the T pM valued vector functions vi satisfy the di erential equations _ vi(t) =Xj ij(t)vj(t) : (37) Since vi(0) 2 D( p), we can conclude that vi(t) 2 D( p) for all t 2 I. (Indeed, if is any linear functional on T pM that vanishes on D( p), then the scalar functions i(t) = h ; vi(t)i satisfy the linear system _ i = Pj ij j. Since the i vanish for t = 0, they vanish for all t, and therefore h ; vi(t)i = 0 for all t 2 I.) This shows that pe tYXietY 2 D( p) for i = 1; : : : ; k, t 2 I. If we let q = pe tY , and observe that the vectors qXi form a basis of D(q), we have established that vetY 2 D( p) whenever v 2 D(q), if q = pe tY , and t belongs to an interval I( p) that may depend on p. This in particular applies to p. We now show that ve(t1 t2)Y 2 D( (t1)) whenever v 2 D( (t2)), with no restriction on t other than the requirement that t1; t2 2 J . To see this, write t1 t2 if ve(t1 t2)Y 2 D( (t1)) whenever v is in D( (t2)), i.e. if the linear transformation t1;t2 : T (t2)M ! T (t1)M given by t1;t2(v) = ve(t1 t2)Y actually maps D( (t2)) into D( (t1)). It is clear that t1 t2 implies t2 t1, since the spaces D( (t1)) and D( (t2)) have the same dimension. Also, trivially, t1 t1, and the transitivity property (t1 54 WENSHENG LIU AND H ECTOR J. SUSSMANN t2) ^ (t2 t3) ) (t1 t3) holds as well. So is an equivalence relation on J . If C is an equivalence class, and t 2 C, then we know that there is an open interval L containing 0 such that, if q = (t)e sY , s 2 L, then vesY 2 D( (t)) for v 2 D(q). But this says that t t s for s 2 L. So t is an interior point of C. Therefore C is open. Since all the equivalence classes modulo are open, and J is connected, we conclude that J itself is an equivalence class, i.e. that t1 t2 for all t1, t2 in J , which is precisely the desired conclusion. PROOF OF LEMMA 3. After replacing M with a neighborhood of p, we may assume that D is spanned by a k-tuple X1;X2;X4; : : : ;Xk+1 of smooth vector elds, where X1 = X;X2 = Y , and that there are smooth vector elds X3;Xk+2; : : : ;Xn such that fXk(x); k = 1; : : : ; ng is a basis of TxM for every x 2M . Let be the map de ned by (x1; : : : ; xn) = pexnXn ex1X1 : Since X1; : : : ;Xn are linearly independent at p, is well de ned on a neighborhood of the origin in IRn, and maps some cube Cn( ) di eomorphically onto a neighborhood U of p in M . We choose such that in addition . The inverse map 1 de nes a chart, by means of which we identify U with Cn( ), so that p becomes the point 0. Clearly, X1 just equals @ @x1 . Moreover, X2(x) is equal to @ @x2 at all points x such that x1 = 0. So X2 = @ @x2 + x1Pni=1 i @ @xi , where the i are smooth functions on Cn( ). We now replace M by U and by its restriction to the interval ( ; ). We now have to show that 3 is of the form (25). For this we need only show that 3 = 0 when x1 = x3 = x4 = = xn = 0. Let = f(0; x2; 0; : : : ; 0) : jx2j < g. Since [X1;X2] = Pni=1( i+x1 @ i @x1 ) @ @xi , it su ces to show that the coe cient of @ @x3 in the above expression for [X1;X2] is identically zero on , i.e. that, on , [X1;X2] is a linear combination of the vector elds @ @xi , i 6= 3. To show this, we rst SUB-RIEMANNIAN METRICS 55 observe that pex2X2 @ @xi = pXiex2X2 for i 2 : (38) Let I = f1; 2; 4; : : : ; k + 1g. Then, if i 2 I, the vector eld Xi is a section of D. Assume now that i 2 I and i 2. It is clear that the integral curve of X2 passing through p in is exactly the line segment in Cn( ), i.e. the curve . Therefore D is invariant under X2 along . We conclude that qe x2X2Xiex2X2 belongs to D(q) for every q 2 , i 2 I. Letting q = pex2X2, i > 1, we nd that pXiex2X2 2 D(pex2X2), i.e. that @ @xi (q) is in D(q) for all q 2 , i 2 I, i 2. The conclusion is also true for i = 1, since X1 = @ @x1 . So the k vectors @ @xi (q), i 2 I, belong to D(q) for all q 2 . These k vectors are obviously linearly independent, and therefore form a basis of D(q), since D is k-dimensional. Since X1 is a section of D, and D is X2-invariant along , the vector [X1;X2](q) belongs to D(q) for q 2 . This implies in particular that [X1;X2](q) is a linear combination of the @ @xi (q) for i 2 I. Since 3 = 2 I, this establishes the desired conclusion about [X1;X2]. Finally, we remark that, if [X; [X;Y ]](p) 62 D(p), then we can choose X3 to be [X; [X;Y ]]. With this choice, X3 = [X1; [X1;X2]] = n Xi=1 2@ i @x1 + x1@2 i @x21 @ @xi : (39) Since X3(p) = @ @x3 (0), we conclude that @ 3 @x1 (0) = 1 2 6= 0. We are now ready to state the Normal Form Theorem. Recall that (M;E;G) is the set of all E-arcs in M such that _ (t) has G-length 1 for almost all t in the domain of . If U is an open subset of M , we can restrict E to U , obtaining a two-dimensional distribution EdU on U , and then we can de ne the restriction Gd(EdU) of G to EdU . We will simply write (U;E;G) rather than (U;EdU;Gd(EdU)). If U is such that there is an orthonormal basis (X;Y ) of sections of EdU , then (U;E;G) is exactly the set of trajectories of the control system _ x = uX(x) + vY (x) ; u2 + v2 1. 56 WENSHENG LIU AND H ECTOR J. SUSSMANN Theorem 4 Let (M;E;G) be a sub-Riemannian manifold such that dimM 3 and dimE = 2. Let : [a; b]! M be a regular abnormal extremal. Then for every t 2 [a; b] there exist a > 0, an open neighborhood V of ( t), a cubic coordinate chart of radius , centered at ( t), with domain V , and an orthonormal basis (X;Y ) of sections of EdV , such that (i) X and Y are given, relative to the chart , by (34), where the k are smooth functions such that 3 is of the form (25) and @ 3 @x1 (0) 6= 0. (ii) there is an " > 0 such that ([ t "; t + "] \ [a; b]) V and (s) = (0; s t; 0; : : : ; 0) for s 2 [ t "; t+ "] \ [a; b]. Conversely, if is an arc contained in the domain V of a cubic chart , (X;Y ) is an orthonormal basis of sections of E on V such that (i) holds, and is of the form s ! (0; s; 0; : : : ; 0), then (a) is an abnormal extremal of E, and (b) is strictly abnormal if and only if the function 2 cannot be expressed along as a linear combination of 4; : : : ; n with constant coe cients. PROOF. If is the projection of a regular abnormal biextremal : [a; b]! RA(E), then we can extend to a regular abnormal biextremal ext de ned on a larger interval [a ; b+ ], and let ext = M o ext, so ext is a regular abnormal extremal de ned on [a ; b+ ]. We may then replace by ext. This means that we can assume that t is an interior point of [a; b]. Also, without loss of generality, we may assume that t = 0. We pick > 0 such that a , b, and replace by its restriction to [ ; ]. In addition, will also be required to satisfy two more conditions |labeled (C1), (C2)| to be stated later. Let p = (0). In view of Theorem 3, we may assume that (C1) is a nice abnormal extremal. SUB-RIEMANNIAN METRICS 57 Let U , Y , ! be such that the four conditions of De nition 16 are satis ed. We may also assume that (C2) ([ ; ]) U , and that there is a smooth section X of E such that (X;Y ) is an orthonormal basis of Sec(E;U). Let D = ker!. Since ! is nowhere vanishing on U , D is a distribution of dimension n 1 on U . Since LY ! 0 along , we have !([Y;Z]) = Y !(Z) (LY !)(Z) = Y !(Z) (40) along for any smooth vector eld Z on U . Applying (40) to an arbitrary section Z 2 Sec(D) |in which case !(Z) vanishes along | we conclude that the distribution D is invariant under Y along . Applying (40) to Z = X, we nd that !([X;Y ]) vanishes along , so [X;Y ](q) 2 D(q) for every q 2 ([a; b]). Applying (40) to Z = [X;Y ] we conclude that !([Y; [X;Y ]] also vanishes along . Since !(q) does not annihilate E3(q) for any q 2 U , it follows that !([X; [X;Y ]])( (t)) 6= 0 for all t. Therefore [X; [X;Y ]](q) = 2 D(q) for q 2 ([a; b]), and we may assume |after shrinking U and , if necessary| that [X; [X;Y ]](q) = 2 D(q) for all q 2 U . We have therefore shown that D has almost all the properties of Lemma 3. The only remaining problem is that X;Y need not be sections of D. To take care of this, we let (Z1; : : : ; Zn 1) be a global basis of sections of D, chosen in such a way that Z1 = X on and Z2 = Y on . (This may require shrinking U and further.) Since [X; [X;Y ]](q) 62 D(q) for q 2 U , the vectors Z1(q); : : : ; Zn 1(q); [X; [X;Y ]](q) span TqM for all q 2 U . So we can write X = n 1 Xi=1 iZi + [X; [X;Y ]] and Y = n 1 Xi=1 iZi + [X; [X;Y ]] ; (41) where i; i; ; are smooth functions. Then, since X(q) and Y (q) belong to D(q) for q 2 ([a; b]) but [X; [X;Y ]](q) 62 D(q), we have 58 WENSHENG LIU AND H ECTOR J. SUSSMANN ( (t)) = ( (t)) = 0 for all t 2 [a; b]. Since X = Z1 and Y = Z2 on , we may assume (after shrinking U further, if necessary) that X;Y;Z3; : : : ; Zn 1 are linearly independent everywhere. Therefore if we let D be the distribution on U spanned by X;Y;Z3; : : : ; Zn 1, then D is a distribution of dimension n 1. Clearly, D(q) = D(q) for q 2 ([a; b]). This implies, in particular, that [X; [X;Y ]](q) 62 D(q) for q 2 ([a; b]). We now show that D is invariant under Y along . If Z is an arbitrary smooth section of D, then we can write Z = 1X + 2Y + n 1 X k=3 kZk : But then [Y;Z] = 1[Y;X] + n 1 X k=3 k[Y;Zk] + (Y 1)X + (Y 2)Y + n 1 X k=3(Y k)Zk : Since D is Y -invariant along , the vectors [Y;Zk](q) belong to D(q) for every q 2 ([a; b]). The Zk(q) also belong to D(q), and we know that X(q), Y (q) and [Y;X](q) are in D(q) as well. So [Y;Z](q) 2 D(q) = D(q) for q 2 ([a; b]), and D is invariant under Y along . We can now apply Lemma 3, and choose a cubic coordinate chart (x1; : : : ; xn) of radius with respect to which (i) holds. The existence of " for which (ii) holds then follows from Remark 12. We now prove the converse. The Hamiltonian of the control system _ x = uX(x) + vY (x) is H(x; ; u; v) = H(x1; : : : ; xn; 1; : : : ; n; u; v) = 1u+ 2v + vx1 n Xj=1 j j(x) : The adjoint equations along say _ 1 = Pnj=1 j j( (t)), _ k = 0 for k = 2; : : : ; n. Since 3 vanishes along , these equations are satis ed by 1 = 2 = 0, 3 = 1, 4 = : : : = n = 0. With this choice of the i, H( (t); ; u; v) vanishes for all (u; v). So is an abnormal extremal. SUB-RIEMANNIAN METRICS 59 We now seek conditions that will guarantee that is not a normal extremal. For to be a normal extremal, there would have to be a solution t ! (t) of the adjoint equations with the property that for each t the function (u; v) ! H( (t); (t); u; v) is minimized on the unit disc by u = 0, v = 1, and the minimum value is 1. Since H( (t); (t); u; v) = 1u+ 2v, this will happen if and only if 1 = 0 and 2 = 1. But then the rst adjoint equation says that 2 = Pni=4 i i. So is a normal extremal if and only if there are constants 4; : : : ; n such that 2 = Pni=4 i i along . 7.3 The optimality theorem We are now ready to prove: Theorem 5 If (M;E;G) is a sub-Riemannian manifold such that dimM 3 and E is a two-dimensional distribution, then every regular abnormal extremal is locally uniquely time optimal. PROOF. We will show that the interval [a; b] can be covered by open intervals I such that the restriction of to every closed subinterval of an I is uniquely optimal. Once this is proved, the local optimality of follows by letting > 0 be a Lebesgue number of the covering fI g. If a t1 < t2 b, and t2 t1 , then the interval [t1; t2] is contained in one of the I , and therefore the restriction of to [t1; t2] is uniquely optimal. To prove the existence of the I , it su ces to pick a t 2 [a; b] and nd a > 0 such that the restriction of to the interval [a; b]\ [ t ; t+ ] is uniquely time-optimal. Theorem 4 says that there exist a neighborhood V of ( t), a chart on V centered at ( t), and an " > 0, such that (i) and (ii) of Theorem 4 hold. This means that we can apply Lemma 2, and conclude that the restriction of to some neighborhood of t is uniquely optimal. 60 WENSHENG LIU AND H ECTOR J. SUSSMANN 8 Strict abnormality We now discuss the extra conditions that are needed for a regular abnormal extremal to be strictly abnormal, and show in particular that most regular abnormal extremals are in fact strictly abnormal. In Theorem 4 we have already characterized strict abnormality for abnormal extremals in normal form. An alternative characterization can be obtained in terms of conditions on vector elds, as we now explain. Use adg to denote the map X ! [g;X] on the space of smooth vector elds. Let (f; g) be an orthonormal basis of sections of E on an open set U and let : [a; b]! U be an integral curve of g. We make the following regularity assumption (REG) there exists an integer k > 0 such that, along , the vector eld adk+1 g (f) is a linear combination with smooth coe cients of g and the adjg(f) for 0 j k. Write adk+1 g (f)( (t)) = k X j=0 j(t)adjg(f)( (t)) + (t)g( (t)) ; (42) where and the j are smooth functions of t. Let be a eld of covectors along such that = ( ; ) satis es _ (t) = ! Hg( (t)). Then the derivative along of any function of the form HX , for X a smooth vector eld on U , is the function H[g;X]. Let 'j(t) = Hadjg(f)( (t)), and write (t) = Hg( (t)). Then the 'j satisfy _ 'j = 'j+1, while _ = 0. Since 'k+1 = 0'0 + : : :+ k'k + , we see that the vector ('0; : : : ; 'k; ) satis es a linear homogeneous system of di erential equations. Therefore this vector vanishes identically if it vanishes at one point. So, if for some t 2 [a; b] there exists a nonzero 2 T (t)M such that h ; g( (t))i = 0 and h ; adjg(f)( (t))i = 0 for j = 0; : : : ; k, then the integral curve of ! Hg that goes through ( (t); ) SUB-RIEMANNIAN METRICS 61 at time t is an abnormal biextremal, and it is clear that the projection of is . We now investigate whether one can also nd a normal biextremal that projects onto . The arc would have to be an integral curve of !H , where H is the Hamiltonian of (M;E;G), i.e. the function (x; ) ! 12 h ; f(x)i2 + h ; g(x)i2 . Since !H = Hf ! Hf Hg ! Hg, d M( _ (t)) = _ (t), d M ( ! HX) = X for every X, and _ (t) = g( (t)), we see that Hf 0 and Hg 1 along , so must be an integral curve of ! Hg. As before, the functions 'j satisfy _ 'j = 'j+1, while is now equal to 1. Since '0 0, we see that 'j 0 for all j. In particular, 'k+1 0. Since 'k+1 = 0'0 + : : :+ k'k + , we see that 0. Since 1, we conclude that 0. So is strictly abnormal if (t) 6= 0 for some t. Summarizing, we have proved: Theorem 6 Let (M;E;G) be a sub-Riemannian manifold such that E is two-dimensional. Let (f; g) be an orthonormal basis of sections of E on an open subset U of M , and let : I ! U be an integral curve of g. Then: (i) if the regularity condition (REG) holds, and in addition for some t 2 I the linear span of g( (t)) and the vectors adjg(f)( (t)) for 0 j k is not the whole tangent space T (t)M , then is an abnormal extremal; (ii) if the coe cient of (42) does not vanish identically, then is strictly abnormal. 9 Some special cases We now make a more detailed analysis of the abnormal extremals in some special cases. Speci cally, we will consider regular distributions, low-dimensional manifolds, and a Lie group example. For the cases when dimM = 4 or 5 we will assume that E is a regular distribution. In the three-dimensional case, if E is regular, then the only abnormal 62 WENSHENG LIU AND H ECTOR J. SUSSMANN extremals are the constant trajectories, so the only interesting case is when E is nonregular at some points. We will assume that the singularities of E occur in a generic way. (The singularities of generic two-dimensional distributions in three-dimensional manifolds are well understood, cf. e.g. [29].) 9.1 2and 3-generating distributions The following is a direct consequence of the results of the previous sections: Theorem 7 Let (M;E;G) be a sub-Riemannian manifold such that E is two-dimensional. Then: (i) if E is 1-generating (in which case, of course, dimM = 2), then (M;E;G) is in fact Riemannian, all extremals are normal, and the extremals are exactly the ordinary Riemannian geodesics; (ii) if E is 2-generating (in which case dimM 3 and E is strongly bracket-generating), then there are no nonconstant abnormal extremals, and all the minimizers are normal extremals; (iii) if E is 3-generating (in which case dimM 5), then RA(E) = (E2)?\T#M , and all locally simple abnormal extremals parametrized by arc-length are regular. In all three cases, all locally simple extremals parametrized by arc-length are smooth and locally optimal. 9.2 3-Regular distributions Let E be a smooth k-regular distribution on a manifold M . For each p 2 M , and each integer k such that 1 k k, de ne LkE(p) = Ek(p)=Ek 1(p). We can then de ne bilinear maps Qk;j E (p) : LkE LjE ! Lk+j E by letting Qk;j E (p)(v;w) = z if f 2 Seck(E), g 2 Secj(E), p 2 Domain(f)\Domain(g), v = f(p) mod Ek 1(p), v = g(p) mod Ej 1(p), and z = [f; g](p) mod Ek+j 1(p). (It is easy to see that z does not depend on the choice of f and g. Indeed, let r = dimEk 1(p), s = dimEk(p). Let (Z1; : : : ; Zn) be a basis of sections of TM , chosen so SUB-RIEMANNIAN METRICS 63 that (Z1; : : : ; Zr) is a basis of sections of Ek 1 and (Z1; : : : ; Zs) is a basis of sections of Ek. If f1; f2 2 Seck(E), f1(p) f2(p) 2 Seck 1(E), and g 2 Secj(E), then we can write fi = Ps̀=1 ìZ` near p, where the ì are smooth functions such that 1̀(p) = 2̀(p) for r < ` s. Then [f1 f2; g] = Ps̀=1( 1̀ 2̀)[Z`; g] Ps̀=1 g( 1̀ 2̀)Z`. Therefore [f1 f2; g](p) = Pr̀=1( 1̀(p) 2̀(p))[Z`; g](p) Pǹ=1(g 1̀(p) g 2̀(p))Z`(p). The brackets [Z`; g] are sections of Ek+j 1 for ` r, and the Z` are sections of Ek+j 1 for ` s. So [f1 f2; g](p) 2 Ek+j 1(p). This shows that the value of [f; g](p) modulo Ek+j 1(p) does not depend on f but only on the value of f(p) modulo Ek 1(p). A similar argument shows that [f; g](p) modulo Ek+j 1(p) only depends the the value of g(p) modulo Ej 1(p).) Now suppose that (M;E;G) is a sub-Riemannian manifold such that dimM 3, and the distribution E is two-dimensional and 3regular. Then the space L2E(p) is one-dimensional for each p, so L2E is a line bundle. The maps Q1;1 E (p) : E(p) E(p) ! L2E(p) are skewsymmetric and nonzero. So each Q1;1 E (p) induces a map ~ Q1;1 E (p) from the exterior power ^2E(p) to L2E(p), which is obviously an isomorphism. Clearly, ^2E(p) has a natural inner product induced by the inner product G(p) on E(p), so this inner product is also well de ned on L2E(p). Therefore L2E(p) has two distinguished elements, namely, the vectors of norm one, corresponding to the two orientations of E(p). Furthermore, once we have a distinguished element w of L2E(p), then we can use it to de ne a map v ! Q2;1 E (p)(v;w) from E(p) to L3E(p). If (f; g) is an orthonormal basis of sections of E near p, then the class of [f; g](p) modulo E(p) is the corresponding distinguished element of L2E(p). If ( ~ f ; ~ g) is another orthonormal basis, then [ ~ f; ~ g](p) = "[f; g](p) modulo E(p), where " = 1 depending on whether or not the orientations of the two bases are the same. The map from E(p) to L3E(p) corresponding to (f; g) |which in fact does not depend on the choice of the orthonormal basis (f; g), but only on the orientation of the pair 64 WENSHENG LIU AND H ECTOR J. SUSSMANN (f(p); g(p))| is the one that sends each vector v = af(p) + bg(p) to the vector a[f; [f; g]](p) + b[g; [f; g]](p). If ! is an orientation of E(p), we will use E(!; p) to denote the map from E(p) to L3E(p) corresponding to !. Notice that the maps E(!; p) are surjective, and satisfy E(!1; p) = E(!2; p) if !1, !2 are opposite orientations. It is clear that on any open set U on which E has a global orientation !, this orientation can be used to de ne a smooth bundle map from EdU to L3EdU which is surjective on each ber. We now study how the maps E(!; p) relate to the abnormal extremals. We consider separately the two cases that arise depending on whether E3 is of dimension 4 or 5. Assume rst that E3 is four-dimensional, so E is a 3-regular distribution of partial type (2; 1; 1). (For this it of course required that dimM 4.) Then at each point p the kernel of the map E(!; p) is one-dimensional, and independent of the orientation !. So this kernel de nes a line subbundle ~ LE. If (f; g) is an orthonormal basis of sections of E near p, then the ber ~ LE(p) is the set of all vectors of the form af(p) + bg(p) such that a[f; [f; g]](p)+ b[g; [f; g]](p) 2 E2(p). Let = (p; ) be a regular abnormal point in T M . Let (f; g) be a basis of sections of E in a neighborhood U of p. Then we know from Proposition 7 that LE( ) is spanned by the vector XE( ) = a ! Hf ( ) b ! Hg( ), where a = H[g;[f;g]]( ); b = H[f;[f;g]]( ). Since E is of type (2; 1; 1; d1; : : : ; dk), the vector elds [f; [f; g]] and [g; [f; g]] are linearly dependent on U modulo E2. Let , be smooth functions on U such that [f; [f; g]]+ [g; [f; g]] 0 modulo E2 and (p)2+ (p)2 6= 0 for all p 2 U . Then the vector (b; a) is orthogonal to ( (p); (p)), and therefore (a; b) is a multiple of ( (p); (p). Therefore LE( ) is spanned by (p) ! Hf ( )+ (p) ! Hg( ). The image of LE( ) under d M( ) is the one-dimensional subspace of E(p) spanned by (p)f(p)+ (p)g(p), which is precisely the space ~ LE(p) de ned above, and is independent of . So ~ LE(p) = d M (p; )(LE(p; )) for any (p; ) 2 RA(E). (Notice, SUB-RIEMANNIAN METRICS 65 however, that d M (XE(p; )) = d M(XE(p; )). So the canonical orientation of LE does not induce a natural orientation of ~ LE.) It is clear thatM is foliated by the leaves of ~ LE and every regular abnormal extremal is contained in a leaf. Conversely, if : [a; b]! M is an arc which is contained in a leaf of ~ LE, we claim that is a projection of an abnormal biextremal contained in RA(E). Indeed, we can partition the interval [a; b] into intervals Ii = [ti 1; ti] such that each (Ii) is contained in an open set Ui which is the domain of an orthonormal basis (fi; gi) of sections of E and of a coordinate chart i. It is clear that we can choose the basis (fi; gi) in such a way that fi is a section of ~ LE and [fi; [fi; gi]] is a section of E2. If = (p; ) is a regular abnormal point such that p 2 Ui, then we know that LE( ) is spanned by ! Hfi( ). So the vector eld ! Hfi, which is smooth on T Ui, is tangent to (E2)? on the relatively open dense subset RA(E), and therefore is tangent to (E2)? everywhere. Moreover, ! Hfi is tangent to (E3)? as well. (To see this, observe that, since [fi; [fi; gi]] is a section of E2, we can write [fi; [fi; gi]] = 1fi + 2gi + 3[fi; gi] (43) on Ui, where the j are smooth functions. But then [fi; [gi; [fi; gi]]] = [gi; [fi; [fi; gi]]] = (gi 1)fi + (gi 2)gi + (gi 3) 1 [fi; gi] + 3[gi; [fi; gi]] ; (44) so [fi; [gi; [fi; gi]]] is a section of E3. On the other hand, (E3)? is de ned by the four equations h1 = h2 = h3 = h4 = 0, where h1 = Hfi, h2 = Hgi, h3 = H[fi;gi], and h4 = H[gi ;[fi;gi]]. At a point 2 (E3)?, the derivatives in the direction of ! Hfi of the four functions h1; h2; h3; h4 are, respectively, 0, h3( ), h5( ) and h6( ), where h5 H[fi;[fi;gi]] and h6 H[fi;[gi;[fi;gi]]]. Now (43) and (44) imply that h5 is a linear combination of h1, h2 and h3 with smooth coe cients, so h5 vanishes on (E2)?, and h6 is a linear combination of h1, h2, h3 and h4 with smooth coe cients, 66 WENSHENG LIU AND H ECTOR J. SUSSMANN so h6 vanishes on (E3)?. Therefore the derivatives of h1, h2, h3 and h4 in the direction of ! Hfi vanish on (E3)?, and ! Hfi is tangent to (E3)?.) The arc satis es, on Ii, the equation _ (t) = ui(t)fi( (t)), where the function ui is integrable. Consider the di erential equation _ (t) = ui(t) (t)@fi @x ( (t)) ; (45) relative to the chart i. It is clear that if ( ) is any solution of (45) on Ii such that (t) 2 E2(p)? but (t) = 2 E3( (t))? for any t 2 Ii, then the pair ( dIi; ) is an abnormal biextremal contained in RA(E). Moreover, for any given initial condition (ti 1) =  there exists a solution of (45) on the whole interval Ii, since the right-hand side of (45) is linear in . If is any solution of (45), and we let = ( dIi; ), then is a solution of _ (t) = ! Hfi( (t)). Since ! Hfi is tangent to (E2)?, which is a closed submanifold of T M , the curve has to be entirely contained in (E2)? if  2 E2( (ti 1))?. Similarly, ! Hfi is tangent to (E3)?, which is also a closed submanifold of T M , the curve is either entirely contained in (E3)? or disjoint from it. In particular, if  = 2 E3( (ti 1))?, then is entirely contained in RA(E). It then follows easily that is a projection of an abnormal biextremal contained in RA(E). Indeed, we can x a ̂0 2 E2( (a))? E3( (a))?, and solve (45) on I1, obtaining an abnormal biextremal 1 which is contained in RA(E) and projects onto the restriction of to I1. Then, using ( (t1)) as the new initial condition, we can solve (45) on I2, obtaining an abnormal biextremal 2 which is contained in RA(E), projects onto the restriction of to I2, and is such that 2(t1) = 1(t1). This procedure can clearly be continued until we construct an abnormal biextremal : [a; b] ! RA(E) that projects onto . Finally, we observe that our proof yields extra information. In fact, we have established that an abnormal biextremal that is not regular has to be entirely contained in (E3)?. In other words, there cannot exist abnormal biextremals that are regular on part of their interval SUB-RIEMANNIAN METRICS 67 of de nition but go through (E3)? at some time. Let us call a curve in M a totally singular abnormal extremal if it is the projection of an abnormal biextremal which is entirely contained in (E3)?. Then we have proved the following Proposition 11 Let E be a two-dimensional 3-regular distribution of partial type (2; 1; 1) on a manifold M . Then there is a line subbundle ~ LE of E such that, if U is any open subset of M on which E admits a basis (f; g) of sections, and , are smooth functions on U such that [f; [f; g]] + [g; [f; g]] 0 modulo E2 and (p)2 + (p)2 6= 0 for all p 2 U , then the vector eld Z = f + g is a section of ~ LE on U . The regular abnormal extremals of E are exactly the arcs parametrized by arc-length that are contained in a leaf of the foliation induced by ~ LE. Every locally simple abnormal extremal parametrized by arc-length is either regular or totally singular. We now consider the case when E3 is ve-dimensional, so the maps (!; p) are isomorphisms from E(p) to L3E(p) = E3(p)=E2(p). There is a map E from (E2)? onto (E3=E2) that sends a pair (p; ) to the pair (p; ̂), where ̂ is the linear functional on E3(p)=E2(p) induced by restricting to E3(p) and then using the fact that vanishes on E2(p) to pass to the quotient. Since (!; p) is an isomorphism from E(p) to E3(p)=E2(p), we can use it to induce an inner product on E3(p)=E2(p), which enables us to identify E3(p)=E2(p) with its dual. Then the map ~ E;!;p = (!; p) 1 E(p) is well de ned, and sends E2(p)? onto E(p). Let E;p be the composite map obtained by rst applying ~ E;!;p and then a clockwise 90o rotation in E(p). (Here \clockwise" refers to the orientation !. Notice that the rotation changes sign if we reverse !, and ~ E;!;p also changes sign, so E;p does not depend on !.) If (f; g) is an orthonormal basis of sections of E on an open set U , ! is the orientation such that (f; g) is positively oriented, and v 2 E(p), v = v1f(p) + v2g(p), then (!; p)(v) = v1F + v2G, where F;G are, re68 WENSHENG LIU AND H ECTOR J. SUSSMANN spectively, the classes modulo E2(p) of [f; [f; g]](p) and [g; [f; g]](p) The induced inner product on E3(p)=E2(p) is the one that makes (F;G) an orthonormal basis. If (p; ) 2 (E2)?, then ĥ; F i = h ; [f; [f; g]](p)i and ĥ; Gi = h ; [g; [f; g]](p)i. So the vector in E3(p)=E2(p) that corresponds to ̂ under the identi cation of E3(p)=E2(p) with its dual is ĥ; F iF+ĥ; GiG. Hence ~ E;!;p( ) = ĥ; F if(p)+ĥ; Gig(p), and then E;p( ) = ĥ; Gif(p) ĥ; F ig(p), i.e. E;p( ) = h ; [g; [f; g]](p)if(p) h ; [f; [f; g]](p)ig(p) : (46) Now suppose that (p; ) = 2 RA(E). Then d M(XE(p; )) = E(p; ) k E(p; )k : (47) So we have proved Proposition 12 If (M;E;G) is a sub-Riemannian manifold such that E is 3-regular of partial type (2; 1; 2). Then there exists a smooth surjective vector bundle map E : (E2)? ! E with the property that, if v is any unit vector in E(p), and E(p; ) = v, then d M(p; )(XE(p; )) = v. In particular, if p 2M and v is any unit vector in E(p), then there exists a regular abnormal extremal : ( "; ")!M such that (0) = p and _ (0) = v. 9.3 The 4and 5-dimensional cases The results for the fourand ve-dimensional cases are essentially contained in the conclusions of the previous subsection. In dimension 4, Proposition 11 applies, and in addition the set (E3)? \T#M is empty, so there are no totally singular abnormal extremals. Therefore we get the following theorem, already derived in [25]: Theorem 8 If (M;E;G) is a sub-Riemannian manifold such that dimM = 4 and E is a regular two-dimensional distribution, then SUB-RIEMANNIAN METRICS 69 there exists a line subbundle L of E such that the abnormal extremals are exactly the L-admissible arcs. Through every point in M there pass exactly two oriented (or one unoriented) locally simple abnormal extremals parametrized by arc-length. Moreover every locally simple abnormal extremal parameterized by arc-length is regular, so it is locally uniquely optimal. In dimension ve, the type of a regular distribution E can be (2; 1; 2) or (2; 1; 1; 1). In the (2; 1; 2) case Proposition 12 applies. Moreover, it is easy to see that in this case the regular abnormal extremals are the projections on M of the trajectories of a ow on the unit sphere bundle of E. To prove this, we work locally, and use Equations (20) and (21). Let (f; g) be an orthonormal basis of sections of E. Extend the Riemannian metric G from E to the whole tangent bundle by declaring (f; g; [f; g]; [f; [f; g]]; [g; [f; g]]) to be an orthonormal basis. For each x 2 M , v 2 TxM , let ̂x(v) be the unique linear functional 2 T xM such that h ; [g; [f; g]](x)i = hv; f(x)i, h ; [f; [f; g]](x)i = hv; g(x)i, h ; [f; g](x)i = hv; [f; g](x)i, h ; f(x)i = hv; [f; [f; g]](x)i and h ; g(x)i = hv; [g; [f; g]](x)i. Notice that ̂x(v) 2 E2(x)? if and only if v 2 E(p). Suppose a curve t! (x(t); (t)) = (t)! RA(E) is a solution of (20), (21). Reparametrize so that the equations simply read _ x = h ; [g; [f; g]](x)if(x) h ; [f; [f; g]](x)ig(x) ; (48) _ = h ; [g; [f; g]](x)i @f @x(x) + h ; [f; [f; g]](x)i @g @x(x) : (49) Let v(t) = _ x(t). Since (t) 2 E2(x(t))?, we must have ̂x(t)(v(t)) = (t). But then we can plug in ̂x(v) for in the second equation, and express the time derivative of ̂x(t)(v(t)) in the form ~ F (x(t); v(t)), where (x; v)! ~ F (x; v) is smooth in both variables and quadratic in v. But d dt ̂x(t)(v(t)) = @ @x ̂x(v) x=x(t);v=v(t) _ x(t) + ̂x(t)( _ v(t)) (50) 70 WENSHENG LIU AND H ECTOR J. SUSSMANN = @ @x ̂x(v) x=x(t);v=v(t) v(t) + ̂x(t)( _ v(t)) : (51) Since ̂x is invertible, we end up with an expression for _ v(t) of the form F (x(t); v(t)), where F is a smooth function of x and v, quadratic in v. Then the equations of the reparametrized abnormal biextremals are of the form _ x = v, _ v = F (x; v). To go back to parametrization by arclength, we must rewrite these equations in the form _ x = v kvk , _ v = F (x;v) kvk . If we write w = v kvk , then the rst equation becomes _ x = w. A simple computation shows that _ w = F (x; v) kvk2 hv; F (x; v)iv kvk4 (52) and then _ w = F (x;w) hw;F (x;w)iw, since F (x; v) is quadratic in v. So we have proved Theorem 9 Let (M;E;G) be a sub-Riemannian manifold such that dimM = 5 and E is a regular distribution of type (2; 1; 2). Then there exists a smooth ow on the unit sphere bundle f(x;w) : x 2M ; w 2 E(x) ; kwkG = 1g such that, (i) whenever t ! (x(t); w(t)) is a trajectory of this ow then t ! x(t) is a regular abnormal extremal and w(t) = _ x(t), (ii) every regular abnormal extremal arises in this way. In addition, all the locally simple abnormal extremals that are parametrized by arc-length are regular, so they are locally uniquely optimal and are projections on M of trajectories of . In the type (2; 1; 1; 1) case, we can apply Proposition 11. We know that the regular abnormal extremals are contained in leaves of ~ LE, but we don't yet have such knowledge about the totally singular abnormal extremals. We will show that the totally singular abnormal extremals are also contained in leaves of ~ LE, and therefore all locally simple abnormal extremals parametrized by arc-length are regular. Let p 2M , and let (X;Y ) be an orthonormal basis of sections of ~ LE in an open neighborhood U of p, chosen so that Y is a section of ~ LE. We SUB-RIEMANNIAN METRICS 71 show that every locally simple abnormal extremal that is contained in U and is parameterized by arc-length is an integral curve of either Y or Y . Now, let : [a; b]!M be a locally simple abnormal extremal that is parametrized by arc-length and is contained in U . We already know that is either regular or totally singular, and our conclusion follows trivially if is regular, so we will assume that is totally singular. Then _ (t) can be expressed as _ (t) = u(t)X( (t)) + v(t)Y ( (t)) for almost all t 2 [a; b], where u and v are measurable functions on [a; b] such that u(t)2+ v(t)2 = 1. Let be a nonzero eld of covectors along such that = ( ; ) is a totally singular abnormal biextremal. Write X1 = X;X2 = Y;X3 = [X;Y ];X4 = [X; [X;Y ]];X5 = [Y; [X;Y ]];X6 = [X; [X; [X;Y ]]];X7 = [X; [Y; [X;Y ]], and let 'k(t) = HXk ( (t); (t)). Then, since (t) annihilates E3( (t)) for all t 2 [a; b], we have 'k(t) 0 for k = 1; 2; 3; 4; 5. On the other hand, since [Y; [X;Y ]] is a section of E2, X7 is a section of E3, and then '7 vanishes along . Now _ '4(t) = u(t)'6(t) + v(t)'7(t), so 0 = _ '4(t) = u(t)'6(t). Since (t) never vanishes, and the seven vector elds Xj span the tangent space at each point, we conclude that '6(t) 6= 0 for all t. But then u(t) 0, so _ (t) = v(t)Y ( (t)) for almost all t 2 [a; b], and then is contained in a leaf of the foliation induced by ~ LE, so is in fact a regular abnormal extremal as well. So we have proved a result very similar to the derived eariler for the four-dimensional case: Theorem 10 Let (M;E;G) be a sub-Riemannian manifold such that dimM = 5 and E is two-dimensional and regular of type (2; 1; 1; 1). Then there exists a line subbundle ~ LE of E such that the abnormal extremals are exactly the ~ LE-arcs. Every locally simple abnormal extremal parametrized by arc-length is regular. Through every point in M there pass exactly two oriented (or one unoriented) locally simple abnormal extremals parametrized by arc-length. All the locally simple abnormal extremals parametrized by arc-length are locally uniquely optimal. 72 WENSHENG LIU AND H ECTOR J. SUSSMANN 9.4 The three-dimensional case Let (M;E;G) be a sub-Riemannian manifold such that dimM = 3 and E is a two-dimensional 3-generating distribution. Then it easy to see that, if the set SE = fp 2 M : dimE2(p) = 2g is nonempty, then it is a smooth two-dimensional submanifold of M . Moreover, the tangent space of SE at a p 2 SE and the space E(p) are transversal, so the intersection ~ LE(p) = E(p) \ TpSE is one-dimensional. Then ~ LE is a smooth line subbundle of TSE. Clearly, a nonzero 2 T pM cannot annihilate E2(p) unless p 2 SE. So all the nonconstant abnormal extremals of E are contained in SE. Since these extremals must have at each point a direction that is tangent to SE and belongs to E, we see that every nonconstant abnormal extremal is contained in a leaf of the foliation ~ FE of SE induced by ~ LE. Conversely, it is easy to show that every locally simple curve contained in a leaf of ~ FE and parametrized by arc-length is a regular abnormal extremal. Therefore the following result holds. Theorem 11 Let (M;E;G) be a sub-Riemannian manifold such that dimM = 3 and E is two-dimensional and 3-generating. Then either SE = ; or SE is a two-dimensional smooth surface in M , and there is a line subbundle ~ LE = (EdSE) \ TSE of TSE such that the abnormal extremals of (M;E;G) are exactly the locally absolutely continuous curves that are contained in the leaves of ~ LE. All the locally simple abnormal extremals parametrized by arc-length are regular, and in particular locally uniquely optimal. Remark 13 It is well known that the singularities of a generic twodimensional distribution E on a three-dimensional manifold M are as follows: the points where dimE2(p) = 2 form a smooth two-dimensional surface SE, that consists of two kinds of points: (a) points p where dimE3(p) = 2, and (b) points p where dimE3(p) = 2. The points of SUB-RIEMANNIAN METRICS 73 the second kind form a discrete subset of SE. Theorem 11 therefore provides, for generic distributions E, a complete description of the abnormal extremals on an open dense set whose complement is at most a discrete set. On the other hand, it is also possible to consider muchmore degenerate distributions E. For example, one can easily construct, for arbitrarily large `, sub-Riemannian manifolds (M;E;G) of dimension 3 with the property that ` is the smallest number such that E (p) = TpM for all p 2 M , on which there are E-arcs that are strictly abnormal extremals and are uniquely locally optimal. For example, let M = IR3 with the usual coordinates x; y; z. Let E be the two-dimensional distribution spanned by f = @ @x and g = (1 x) @ @y +x2k @ @z , where k > 0 is an integer. Let G be the Riemannian metric on E under which (f; g) forms an orthonormal basis. It is easily checked that E is nonholonomic and the smallest number ` such that E`(p) = TpM for all p 2M is 2k + 1. Let : [a; b]! M be a curve given by (t) = (0; t; 0). Then, with an argument similar to the one used in Subsection 2.3, it can be proved that is a strictly abnormal extremal, and is uniquely optimal if b a < 2 2k+1 . 9.5 A Lie group example The preceding discussion makes it easy to produce examples of locally uniquely optimal abnormal extremals for invariant sub-Riemannian metrics on Lie groups. For example, let G be any four-dimensional Lie group whose Lie algebra L has two generators f and g such that (i) f , g, [f; g] and [f; [f; g]] form a basis of L, (ii) [g; [f; g]] belongs to the linear span of f , g, and [f; g], and (iii) [g; [f; g]] does not belong to the linear span of f and [f; g]. (For instance, one can take G = SO(3) IR, and let f = K1 1, g = (K1 + K2) 2, where K1, K2, K3 are generators of the Lie algebra so(3) of SO(3) such that [K1;K2] = K3, 74 WENSHENG LIU AND H ECTOR J. SUSSMANN [K2;K3] = K1 and [K3;K1] = K2, and we are identifying the Lie algebra of G with the direct sum so(3) IR. It is easily veri ed that f , g, [f; g] and [f; [f; g]] are linearly independent, and [g; [f; g]] = 2f g, so all our conditions hold.) Let E be the subbundle of TG spanned by f and g, and de ne a sub-Riemannian structure by letting f and g be an orthonormal basis of sections. Theorem 6 then implies that the integral curves of g are strictly abnormal locally uniquely optimal trajectories. 9.6 A nonsmooth abnormal extremal In view of the preceding discussion, it is clear that the simplest situation involving a regular bracket-generating rank-2 distribution where a nonsmooth abnormal extremal parametrized by arc-length could exist is in dimension 6. We now exhibit such an example. Let M = IR6, and de ne vector elds f , g by letting f = @ x1 , g = @ x2 + x1 @ x3 + x21 @ x4 + x1x2 @ x5 + x21x2 @ x6 . We then let E be the distribution spanned by f and g, and de ne a metric on E by declaring (f; g) to be an orthonormal basis. Clearly, [f; g] = @ x3 + 2x1 @ x4 + x2 @ x5 + 2x1x2 @ x6 , [f; [f; g]] = 2 @ x4+2x2 @ x6 , [g; [f; g]] = @ x5 +2x1 @ x6 , [f; [f; [f; g]]] = [g; [g; [f; g]]] = 0, and [g; [f; [f; g]]] = [f; [g; [f; g]]] = 2 @ x6 . Therefore E is bracket-generating and regular of type (2; 1; 2; 1). Let : IR ! M be the curve given by (t) = (0; t; 0; 0; 0; 0) for t 0 and (t) = (t; 0; 0; 0; 0; 0) for t 0. Then is an E-curve, and a simple computation shows that is a simple abnormal extremal parametrized by arc-length. It is clear that is not smooth, and therefore is not regular. It can be shown that is not locally optimal (cf. Appendix E.) We do not know any examples of locally optimal nonsmooth extremals. SUB-RIEMANNIAN METRICS 75 Appendix A: The Gaveau{Brockett problem We study a sub-Riemannian minimization problem rst considered by Gaveau in 1977 in [9] and then by Brockett in 1981 in [5]. The manifold M for this problem is IRn so(n). Gaveau's Theorem 1, in p. 133, says, in our terminology, that if Z 2 so(n) has rank > 2 then there are no normal extremals joining (0; 0) to (0; Z). If true, this would have implied that the minimum-length arcs from (0; 0) to (0; Z) |whose existence follows by standard arguments| are strictly abnormal extremals, thereby proving, 14 years before Montgomery's 1991 example, that strictly abnormal sub-Riemannian minimizers can occur. However, Brockett explicitly computed, for arbitrary Z 2 so(n), the set N(Z) of all normal extremals from (0; 0) to (0; Z), and showed in particular that N(Z) 6= ;. In addition, he determined the elements of N(Z) of minimum length, and computed their length L(Z). Finally, he stated that \an elementary Lagrange multiplier argument shows that there exists a skew-symmetric matrix such that _ u+ u = 0" ([5], p. 22), which is tantamount to saying that all minimizers are normal extremals. Together, these results imply that L(Z) is the minimum length, and the members of N(Z) of length L(Z) are the minimizers. However, Brockett did not spell out his Lagrange multiplier argument. Moreover, it turns out that the full proof of Brockett's assertion shows the occurrence of a new phenomenon: if n > 2, then there exist abnormal extremals, and even strictly abnormal ones, but the strictly abnormal extremals cannot be minimizers. In view of the existence of con icting claims surrounding Brockett's result, plus the fact that the proof is interesting in its own right, we will now show how to derive Brockett's conclusion, using the Maximum Principle together with a supplementary inductive argument to exclude the possibility of strictly abnormal minimizers. 76 WENSHENG LIU AND H ECTOR J. SUSSMANN Let V be a nite-dimensional linear space over IR, endowed with a Euclidean inner product h ; i. Let Z(V ) be the space of all linear, skew-symmetric maps from V to V . Let n = dim V . Then Z(V ) has dimension n(n 1) 2 . The space Z(V ) is also Euclidean, with the inner product de ned by hA;Bi = Tr(AB) for A;B 2 Z(V ), where Tr( ) is the trace. Clearly V is isomorphic to IRn and Z(V ) is isomorphic to so(n). Moreover, we can identify Z(V ) with the second exterior power ^2V , by means of the canonical isomorphism V : ^2V ! Z(V ) de ned as follows. For v;w 2 V , let v ^ w 2 Z(V ) denote the linear skew-symmetric map on V de ned by letting (v ^ w)(x) def = hw; xiv hv; xiw for x 2 V : Then (v;w) ! v ^ w is a bilinear skew-symmetric map from V V to Z(V ), and therefore induces a linear map V : ^2V ! Z(V ) that sends the exterior product v ^ w 2 ^2V to the skew-symmetric map v ^ w 2 Z(V ). The map V is easily seen to be an isomorphism. Let MV = V Z(V ). Then MV is a nilpotent Lie algebra of dimension n(n+1) 2 . The Lie bracket on MV is de ned by [(x; z); (~ x; ~ z)] = (0; 2x^ ~ x). Via the exponential map,MV can be identi ed with its corresponding connected simply connected Lie group, and the group multiplication on MV is then given by (x; z) (~ x; ~ z) = (x+ ~ x; z+ ~ z+x^ ~ x). As a Lie algebra, MV is generated by V , identi ed with V f0g. If we let fu denote the left-invariant vector eld on MV that corresponds to (u; 0), then fu can be regarded as a map from MV to MV , given by fu(x; z) = (u; x ^ u). It is not hard to see that [fu; fv] = 2gu;v, where gu;v(x; z) = (0; u ^ v). The linear subspace V of MV gives rise to a left-invariant bracket-generating distribution EV , which carries a left-invariant Riemannian metric GV such that hfu; fviGV hu; vi. The Gaveau-Brockett problem is that of determining the structure of the trajectories that minimize length for the sub-Riemannian manifold (MV ; EV ; GV ). The EV -admissible trajectories of (MV ; EV ; GV ) are SUB-RIEMANNIAN METRICS 77 exactly the trajectories of the control system on MV given by _ x = u ; _ z = x ^ u ; (53) where the control set is V . A simple application of Ascoli's Theorem shows that given any pair (p; q) of points ofMV there exists at least one EV -arc such that @ = (p; q) and minimizes length among all the EV -arcs ~ such that @~ = (p; q). Our problem is to understand these length-minimizing arcs. (Gaveau and Brockett actually considered the problem of minimizing the \energy" 1 2 R 1 0 k _ (s)k2ds among all the EV arcs : [0; 1] ! MV such that (0) = p and (1) = q. It is easy to show that this problem is equivalent to ours, in the sense that an arc : [0; 1] ! MV , for which R 1 0 k _ (s)kds = , solves the GaveauBrockett minimization problem if and only if the arc ~ : [0; ] ! MV given by ~ (t ) = (t) for 0 t 1 is parametrized by arc-length and minimizes length.) Since one can always reparametrize an arc by arc-length without changing the length, we will limit ourselves to considering arcs parametrized by arc-length. Finally, in view of the left-invariance of the problem, we may assume without loss of generality that p = 0. Let FV be the control system de ned by the equations (53) with the control constraint kuk2 1. Then the EV -arcs that minimize arclength and are parametrized by arc-length are precisely the minimumtime trajectories of FV . Let U denote the closed unit ball in V . After identifying in an obvious way the dual of MV with MV itself, and the cotangent bundle T MV with MV MV , the Hamiltonian of FV is the function H on MV MV U given by H(x; z; ; ; u) = h ; ui + h ; x ^ ui = h ; ui Tr( :(x ^ u)) = h ; ui + 2hx; ui = h ; ui 2h x; ui ; (54) for x 2 V , z 2 Z(V ), 2 V , 2 Z(V ), u 2 U . The adjoint equations 78 WENSHENG LIU AND H ECTOR J. SUSSMANN are _ = 2 u ; _ = 0 : (55) A trajectory [0; T ] 3 t! (t) = (x(t); z(t)) 2 V Z(V ) corresponding to a control u : [0; T ] ! V is an extremal if there exist a nowhere vanishing absolutely continuous function [0; T ] 3 t ! ( (t); (t)) 2 V Z(V ) and a constant 0 0 such that (a) ( ; ) satis es the adjoint equations (55), and (b) the minimization condition 0 = H(x(t); z(t); (t); (t); u(t)) = min v2U H(x(t); z(t); (t); (t); v) (56) holds for almost all t 2 [0; T ]. If 0 can be chosen so that 0 > 0, then is a normal extremal. In that case, we can always assume 0 = 1 due to homogeneity. If 0 can be chosen equal to 0, then is an abnormal extremal. (As explained before, the two possibilities are not mutually exclusive.) Assume that T > 0 and = (x; z) : [0; T ] ! MV is parametrized by arc-length and is a normal extremal. Let u = _ x, and let ; be a nontrivial solution of (55) such that the minimization condition holds with 0 = 1. It is clear that the minimum value on the ball U of the functional v ! h (t) 2 (t)x(t); vi is kh (t) 2 (t)x(t)k, so kh (t) 2 (t)x(t)k = 1 for a.e. t. Moreover, the minimum is attained for v = (t)+2 (t)x(t). This yields u(t) = (t)+2 x(t) for a:e: t, which further implies that the function u( ), which was only assumed a priori to be measurable and U -valued, is in fact absolutely continuous, so the equality u = + 2 x holds in fact for all t, and _ u = _ + 2 _ x = 4 u. So the control u satis es the equation _ u = u for some 2 Z(V ), and in addition ku(t)k = 1 for all t. Conversely, suppose t ! (x(t); z(t)) = (t) is a trajectory that corresponds to a control u = _ x that satis es _ u = u for some  2 Z(V ) and ku(0)k = 1. Then ku(t)k = 1 for all t, so is parametrized by arc-length. De ne ( ; ) by = 1 4 , (t) = 2 x(t) u(t). Then ( ; ) satis es the adjoint SUB-RIEMANNIAN METRICS 79 equations (55) and the minimization condition (63) holds with 0 = 1, which implies that is a normal extremal. So we have proved the following characterization of normal extremals: Proposition 13 Let T 0. A trajectory = (x; z) : [0; T ] ! MV , corresponding to a control u = _ x is a normal extremal for (MV ; EV ; GV ) parametrized by arc-length if and only if there exists a skew-symmetric linear map  from V to V such that u satis es _ u u and ku(0)k = 1. Using this characterization, we now prove: Theorem 12 All the minimum-length trajectories of (MV ; EV ; GV ) that are parametrized by arc-length are normal extremals. PROOF. We use induction on the dimension n of V . If n = 1, this is trivial. Assume that the conclusion is true for all 1 n < k. Let V be of dimension k. Let = (x; z) : [0; T ] ! MV be a length-minimizing trajectory such that (0 = (0; 0). Let u = _ x be the corresponding control. The Maximum Principle guarantees that is either a normal extremal or an abnormal one. Let us assume that is an abnormal extremal. We will prove that in that case is a normal extremal as well. Since is an abnormal extremal, there exists a nowhere vanishing absolutely continuous function ( ; ) : [0; T ] ! V Z(V ) such that ( ; ) satis es the adjoint equations (55) and in addition 2 x 0 on [0; T ] : (57) In particular (57) implies that (0) = 0. Di erentiating (57) we have 0 = d dt( 2 x) = 4 u, so u 0. But then _ = 2 u 0. Therefore 0 on [0; T ]. The nontriviality of ( ; ) then implies that 6= 0. Let S = ker . Then dimS < k. Since 0, (57) implies that x(t) 2 S for all t, and then u(t) 2 S as well. Therefore = (x; z) 80 WENSHENG LIU AND H ECTOR J. SUSSMANN is a trajectory of (53) with initial condition x(0) = 0, z(0) = 0, corresponding to the S-valued control u( ). This implies that takes values in S Z(S). Clearly also minimizes length and is parametrized by arc-length when regarded as an admissible arc for the sub-Riemannian manifold (MS ; ES; GS). Since S has dimension < k, the inductive hypothesis tells us that is a normal extremal of (MS; ES ; GS). Then Proposition 13 implies that there exists a 2 Z(S) such that _ u u. Clearly, we can extend to a skew-symmetric linear map  on V . Since u(t) 2 S for all t, we have _ u(t) = u(t) for t 2 [0; T ], and  2 Z(V ). But then it follows from Proposition 13 that is a normal extremal. Remark 14 In the above proof, we have shown that all the minimizing abnormal extremals for the Gaveau-Brockett problem that are parametrized by arc-length are normal extremals. If dim V > 2, then it is easy to produce examples of non-minimizing strictly abnormal extremals parametrized by arc-length. For example, pick any nonzero 2 Z(V ) with a nontrivial kernel K, let u : [0; T ] ! K be an arbitrary measurable function such that ku(t)k = 1 for all t, and then let x(t) = R t 0 u(s)ds, z(t) = R t 0(x(s) ^ u(s)) ds, (t) = (x(t); z(t)). If we take (t) 0, then it is easy to see that t ! (x(t); z(t); (t); ) is an abnormal biextremal, so is an abnormal extremal. By choosing u( ) to be discontinuous, we can make sure that is not a normal extremal. If dim V 4, then it is clear that u( ) can be taken to be smooth, and even real-analytic. (It su ces to make sure that u( ) is not a solution of an equation _ u = u for any skew-symmetric map .) Brockett has shown in [5] how Proposition 13 and Theorem 12 can be used to derive, for any Z 2 Z(V ), an explicit formula for a minimumlength trajectory from (0; 0) to (0; Z), and for the sub-Riemannian distance from (0; 0) to (0; Z). SUB-RIEMANNIAN METRICS 81 Appendix B: Proof of Theorem 1 We show that (M;E;G) is the set of trajectories of a control system, and then use the \Pontryagin Maximum Principle" (cf. [2], [13], [15], [20], [26]), which gives a necessary condition for a trajectory of a control system to minimize a cost functional such as time. In our situation, the condition says precisely that is an extremal. We will rst review the de nition of a control system and the statement of the Maximum Principle. For simplicity, we will only consider the case of interest, namely, minimum time problems with xed endpoints, arising from an autonomous control system of class C1, although the Maximum Principle is a much more general result, valid as well for nonautonomous systems with vector elds that are locally Lipschitz but not necessarily of class C1, and for minimization problems with variable endpoints, and cost functionals other than time. B.1: Control systems De nition 18 An autonomous control system of class C1 on a smooth manifold M is a parametrized family F = ffu : u 2 Ug of vector elds of class C1 on M . Remark 15 We emphasize that in the above de nition U is just an arbitrary set, and there is no requirement that U be endowed with extra structure such as a topology or a metric. To such a system F one associates a collection (F ) of trajectories as follows: De nition 19 A control for F is a function : [a; b]! U de ned on some compact interval [a; b]. An F -trajectory generated by a control is an arcz : [a; b]!M such that _ (t) = f( (t); (t)) for almost all t 2 zRecall that arcs are absolutely continuous by de nition. 82 WENSHENG LIU AND H ECTOR J. SUSSMANN [a; b], where we have written f(x; u)def =fu(x). A control is admissible if the time-varying vector eld (x; t)! f (x; t)def =f(x; (t)) arising from satis es the following Carath eodory conditions: (i) f (x; t) is Lebesgue measurable as a function of t for each xed x, (ii) for every chart : V ! IRn of M and every compact subset K of (V ) there exists an integrable function ' : [a; b]! IR such that kf  (x; t)k+ @f @x (x; t) '(t) for all (x; t) 2 K [a; b] ; (58) where f  : (V ) [a; b]! IRn is the representation of f with respect to the chart , i.e. f  (x; t) = d ( 1(x)):f ( 1(x); t). An admissible trajectory of F is an F -trajectory that is generated by an admissible control . We let (F ) denote the set of all admissible trajectories of F . If 2 (F ), and the domain of is the interval [a; b], we write T ( )def =b a. A trajectory 2 (F ) is time-optimal if T ( ) T (~ ) for all 2 (F ) such that @~ = @ . Remark 16 In Control Theory, the manifold M on which the vector elds fu are de ned is known as the state space of the system F = ffu : u 2 Ug, and the set U is known as the \control space," or the \set of control values." The word \control" is used somewhat vaguely to mean either the U -valued \variable" u or a U -valued function t! (t) de ned on an interval. It is customary to introduce control systems by means of expressions such as \the system _ x = f(x; u), x 2M , u 2 U ." This is to be interpreted as another way of referring to the parametrized family of vector elds ff( ; u) : u 2 Ug. The formal expression _ x = f(x; u) gives rise, for each control function [a; b] 3 t ! (t) 2 U , to a non-autonomous ordinary di erential equation _ x = f(x; (t)), whose solutions are the trajectories of the system. In addition, if U is described by some formula, one often uses this formula instead of the expression SUB-RIEMANNIAN METRICS 83 \u 2 U ." As an illustration of the use of this terminology, consider the example studied in Subsection 2.2. The space M = IR3 was given a sub-Riemannian manifold structure by means of a two-dimensional distribution E, spanned by two vector elds f and g. The metric G on E was de ned by declaring f and g to be an orthonormal basis. We can then construct a control system F in the sense of our de nition by letting U be the unit ball in IR2, and associating to each u 2 U a vector eld fu, de ned by fu = u1f+u2g, if u = (u1; u2) 2 U . The system F is then the vector eld family F = ffu : u 2 Ug. It is easy to verify that the set (F ) is exactly (M;E;G), so we have realized (M;E;G) as the collection of trajectories of a control system. Instead of using the somewhat awkward expression \the system F = ffu : u 2 Ug, where U is the unit ball in IR2 and fu = u1f + u2g if u = (u1; u2) 2 U ," we can introduce F as the system _ x = u1f(x) + u2g(x) ; x 2 IR3 ; u21 + u22 1 ; (59) or, using the formulas for f and g, and u, v rather than u1, u2, as the system _ x = u ; _ y = v ; _ z = vx uy ; u2 + v2 1 : (60) In expressions such as (60), the symbols x, y, z are the \state variables," and u, v are the \control variables." From now on, we will always use this Control Theory language to introduce control systems, with the understanding that the precise interpretation of expressions in this language is obtained by translating them into the vector eld formulation used in our de nitionx. xIn Control Theory, it is sometimes desirable to single out a special class of control functions, such as the set of square-integrable, or piecewise continuous, or piecewise smooth, or piecewise constant functions, assuming, in each case, that the control space has whatever extra structure is needed for such a notion to make sense. To allow for this, one has to de ne a control system to be a parametrized family of vector elds together with the speci cation of a class of control functions. In this work, this extra feature is ignored, because we are always taking the class of control functions to be the class of all controls that are admissible in the sense of De nition 19. 84 WENSHENG LIU AND H ECTOR J. SUSSMANN B.2: The Maximum Principle Let F = ffu : u 2 Ug be an autonomous control system of class C1 on a manifold M . The Maximum Principle gives a necessary condition for a 2 (F ) to be time-optimal. To state this condition, we rst introduce the Hamiltonian lift F of F . By de nition, F is the collection of vector elds f ! Hfu : u 2 Ug on T M . In other words, each fu gives rise to a function Hfu : T M |de ned by Hfu(x; ) = h ; fu(x)i| and this function gives rise to a Hamilton vector eld f udef = ! Hfu . The collection of all the f u is the system F . If : [a; b]! U is an admissible control for F , then a trajectory of F for is a pair ( ; ), where is a trajectory of F for , and is a eld of covectors along (i.e. (t) 2 T (t)M for each t 2 [a; b]) such that, if we write (t) = ( (t); (t)), then is absolutely continuous and _ (t) = f (t)( (t)) (61) for almost all t 2 [a; b]. Equivalently, ( ; ) is a trajectory of F for if and only if _ (t) = f( (t); (t)) and _ (t) = (t) @f @x( (t); (t)) (62) for almost all t 2 [a; b]. Here (62) is known as the adjoint equation along ( ; ). In principle, (62) makes sense in coordinates, if we write the components of points and tangent vectors as columns |so that, for instance, f(x; u) is a column, and @f @x(x; u) is a square matrix| and those of covectors as rows. However, since (62) is equivalent to (61), the property characterized in coordinates by (62) is in fact intrinsic. A eld of covectors along a trajectory : [a; b] ! M of F corresponding to an admissible control : [a; b] ! U is called an adjoint vector along ( ; ) if it is a solution of (62), i.e. if the curve : [a; b] ! T M given by (t) = ( (t); (t)) is absolutely continuous and satis es (61) for almost every t. An adjoint vector along ( ; ) is SUB-RIEMANNIAN METRICS 85 minimizing if there exists a constant 0 0 such that 0 = (t)f( (t); (t)) = minf (t)f( (t); u) : u 2 Ug (63) for almost all t 2 [a; b]. Finally, is nontrivial if (t) 6= 0 for every t. (Notice that, since (62) is a linear time-varying O.D.E. for , if (t) 6= 0 for some t, then is nontrivial, i.e. (t) 6= 0 for all t.) With the above de nitions, the Maximum Principle says: Theorem 13 If F is an autonomous control system of class C1 on a manifold M , and is a time-optimal trajectory corresponding to an admissible control , then there exists a nontrivial minimizing adjoint vector along ( ; ). B.3: Proof of Theorem 1 We are now ready to show that Theorem 1 is a consequence of Theorem 13. To apply Theorem 13 to our situation, we must rst produce, for a sub-Riemannian manifold (M;E;G), a control system F on M such that (F ) is precisely the set (M;E;G) de ned before. Recall that in our general formulation of the Maximum Principle the \control space" U was allowed to be an arbitrary set, with no additional structure whatsoever. In particular, we can take U to be the set of all smooth sections u of E that satisfy ku(x)kG 1 for all x 2M . We then de ne fu to be just u, so fu(x) and f(x; u) are simply alternative ways of writing u(x). We let F = ffu : u 2 Ug. It is then easy to see that the admissible trajectories of F are precisely the arcs in (M;E;G). So our control system F has the desired property that (F ) = (M;E;G). If we now apply the Maximum Principle to this system F , we nd that, if : [a; b] ! M is a time-optimal trajectory, corresponding to a control : [a; b] ! U , then there must exist a nontrivial eld of covectors along that satis es, in local coordinates, the equation _ (t) = (t) @ (t) @x ( (t)) ; (64) 86 WENSHENG LIU AND H ECTOR J. SUSSMANN and a nonnegative constant 0 such that the minimization condition 0 = h (t); (t)( (t))i = minfh (t); u( (t))i : u 2 Ug (65) holds for almost all t. Let us write (t) = ( (t); (t)). We will show that is an abnormal extremal if 0 = 0 and a normal one if 0 6= 0. Suppose rst that 0 = 0. Then (65) implies that (t)v 0 for every v 2 E( (t)) such that jjvjjG 1. Therefore (t) annihilates E( (t)), so (t) 2 E?, and then every H 2 HE vanishes along . Moreover, (64), together with _ (t) = (t)( (t)), say that _ (t) = ! H (t), where H (t) is the Hamiltonian of the vector eld (t). Since (t) is a smooth section of E, we see that _ (t) 2 ! HE( (t)). So is an abnormal extremal. Next consider the case when 0 6= 0. We can then multiply by 1 0 and obtain a new nontrivial minimizing adjoint vector that corresponds to 0 = 1. Our goal is now is to prove that is a normal biextremal, and this is a local property (cf. Remarks 18 and 19 below), so we may assume that M is open in IRn and E has a global orthonormal basis (f1; : : : ; fm) of sections. Our trajectory satises _ (t) = (t)( (t)), where each (t) is a vector eld in U . Write (t)(x) = Pmk=1 k(x; t)fk(x), so the k are functions on M [a; b], smooth in x for each t, measurable in t, and such that Pk k(x; t)2 1 for all (x; t). The minimization condition then says that (t)( (t)) minimizes the function u! h (t); u( (t))i on U , i.e. that the functional t : v = (v1; : : : ; vm)!Xk vkh (t); fk( (t))i ; (66) restricted to the unit ball of IRm, attains its minimum value for the m-tuple ( 1( (t); t); : : : ; m( (t); t)). From this it clearly follows that k( (t); t) = h (t); fk( (t))i. On the other hand, the adjoint equation says that _ (t) = (t) @ @x (t)(x) x= (t) (67) SUB-RIEMANNIAN METRICS 87 = (t) @ @x Xk k(x; t)fk(x) x= (t) (68) = Xk k( (t); t) (t) @fk @x (t) @ @x Xk k(x; t)h (t); fk( (t))i! x= (t) (69) = Xk h (t); fk( (t))i (t) @fk @x (t) @ @x Xk k(x; t)h (t); fk( (t))i! x= (t) : (70) Now, we know that t is minimized on the unit ball of IRm by taking v = 1( (t); t); : : : ; m( (t); t) . On the other hand, for each x 2 M the m-tuple v(x) = ( 1(x; t); : : : ; m(x; t)) belongs to the unit ball. Therefore the function x ! Pk k(x; t)h (t); fk( (t))i has a minimum at x = (t). It follows that @ @x Xk k(x; t)h (t); fk( (t))i! x= (t) = 0 : (71) So _ (t) =Xk h (t); fk( (t))i (t) @fk @x (t) : (72) On the other hand, it follows from the equalities _ (t) = (t)( (t)), (t)( (t)) = Pk k( (t); t)fk( (t)), and k( (t); t) = h (t); fk( (t))i, that _ (t) = Xk h (t); fk( (t))ifk( (t)) ; (73) It is clear that (73) and (72) are precisely the equations of a trajectory of the Hamiltonian H of (E;G). So is a normal biextremal of (E;G), and therefore is a normal extremal of (E;G). Remark 17 The above proof can be slightly simpli ed in the special case when E has a global orthonormal basis (f1; : : : ; fm) of sections. 88 WENSHENG LIU AND H ECTOR J. SUSSMANN Indeed, in that case one can take U to be the closed unit ball in IRm, de ne fu = u1f1 + : : : + umfm for u = (u1; : : : ; um) 2 U , and let F = ffu : u 2 Ug, so one gets a control system with a much smaller control space than the one we have considered. The proof in this case is identical to the one given above, except that the vector eld (t) is now replaced by Pk k(t)fk, so the second terms of (69) and (70) do not occur, and the extra step (71) is not needed to derive (72). The general form of Theorem 1 can then be derived from the one for the special case, by showing that one can always choose a basis of smooth sections of E on some neighborhood of . (It is clear that can be assumed to begin with to be a curve without double points.) Remark 18 As opposed to the Riemannian case, it is not possible to prove Theorem 1 by an argument which is purely local in time, e.g. by showing that satis es the desired conclusion on su ciently short time intervals. The reason for this is that being an extremal is not a local property of trajectories . Indeed, suppose : [a; b] ! M is \locally extremal," i.e. such that the restrictions j : Ij ! M of to intervals Ij that cover [a; b] are extremals. This means that for each j there is a eld of covectors j along j such that ( j; j) is a biextremal. However, there is no reason why the j should match, making it possible to construct a global on [a; b] such that ( ; ) is a biextremal. (Here again the intuitions arising from the Riemannian case are misleading. In the Riemannian case, one can easily show that the covector is uniquely determined by , so any two j 's have to coincide on the intersection of their domains, and then the j's give rise to a global , so the equation of Riemannian geodesics can be proved by purely local arguments.) It follows that, in order to prove Theorem 1 from the Maximum Principle without assuming the existence of a global basis of sections, one needs a more subtle argument, such as the one given in the text or the one of Remark 17. SUB-RIEMANNIAN METRICS 89 Remark 19 Although extremality is not a local property, biextremality is. That is, a curve : I ! T M is a biextremal i for every t 2 I there is an " > 0 such that the restriction of to I \ (t "; t+ ") is a biextremal. Appendix C: Local optimality of normal extremals For completeness, we provide a direct control-theoretic proof of the local optimality of normal extremals. Let (M;E;G) be a sub-Riemannian manifold. Let : (a; b)! M be a normal extremal parametrized by arc-length. Let be a eld of covectors along such that = ( ; ) is a normal biextremal. (For simplicity we assume that the domain of is an open interval. Otherwise we can always extend to a normal biextremal de ned on a slightly larger open interval.) Let H be the Hamiltonian associated to (E;G). Without loss of generality we may assume that is normalized, i.e. H( (t); (t)) = 12 for all t 2 (a; b). To show that is locally optimal, we need only to prove that every c 2 (a; b) has a neighborhood Jc = (c "; c + ") (a; b) such that the restriction of to Jc is optimal. Fix a c 2 (a; b). Let p = (c); = (c). Pick a smooth hypersurface S of dimension n 1 in M passing through p such that vanishes on Tp S. Let  be a smooth 1-form on an open neighborhood of p such that (p ) = , (p) annihilates TpS andH(p; (p)) = 1 2 for all p 2 S\ . For later use we assume that is small enough so that there exists an orthonormal basis (X1; : : : ;Xm) of sections of E on , where m is the dimension of E, and also that is the domain of a coordinate chart. Then H(x; ) = 1 2 m X k=1h ;Xk(x)i2 : (74) 90 WENSHENG LIU AND H ECTOR J. SUSSMANN For each p 2 \ S, let p = ( p; p) be the solution of _ (t) = ! H ( (t)); (c) = (p; (p)). Then clearly = p . Since _ (c) 62 Tp S (because _ (c) = Pmj=1h (c);Xj(c)iXj(c), so that h (c); _ (c)i = Pmj=1h (c);Xj(c)i2 = 1), we can apply the Implicit Function Theorem to conclude that, if " > 0 is small enough and W is a su ciently small open neighborhood of p in S, then the map : (c "; c+ ") W 3 (t; p)! p(t) 2M maps (c "; c+") W di eomorphically onto an open neighborhood U of p in M . We may assume that U is contained in . De ne a smooth function V : U ! IR and a 1-form ! on U by letting V (x) = t and !(x) = p(t) if x = p(t). We will show that dV ! on U . For this purpose, let X be the vector eld on U given by X(x) = d M( p(t)) !H ( p(t)) if x = p(t). Then p satis es _ p(t) = X( p(t)). Let ! HX be the Hamiltonian lift of X. We claim that ! HX( p(t)) = ! H ( p(t)) for p 2 W , t 2 (c "; c + "). Indeed, let k(x) = h!(x);Xk(x)i for k = 1 : : : ;m, if x 2 U . Then X = Pmk=1 kXk on U . Since H is constant along integral curves of ! H , we have H( p(t)) = 1 2 for (t; p) 2 (c "; c + ") W . So Pmk=1 k(x)2 = 1 for x 2 U . It follows from (74) that ! H (x; ) = Pmk=1h ;Xk(x)i ! HXk(x; ). In particular !H ( p(t); p(t)) = Pmk=1 k( p(t)) ! HXk( p(t); p(t)). On the other hand, it is clear that ! HX(x; ) = Pmk=1 k(x) ! HXk(x; ) Pmk=1h ;Xk(x)i ! k (x; ). Since Pmk=1 k(x)2 = 1, it follows that Pmk=1 k(x) ! k(x; ) 0. Therefore ! HX( p; p) = m X k=1 k( p) ! HXk ( p; p) m X k=1h p;Xk( p)i ! k ( p; p) = m X k=1 k( p) ! HXk ( p; p) m X k=1 k( p) ! k( p; p) = ! H ( p; p) ; where p, p are evaluated at t. So p is also a solution of _ = SUB-RIEMANNIAN METRICS 91 ! HX( ); (c) = (p; (p)) for p 2 W . Let (p; t) ! t(p) be the ow of X on U , so that _ t(p) = X( t(p)) and c(p) = p for p 2 U . Then it is clear that p(t) = t(p) for p 2 W . For each t 2 (c "; c + "), let Wt = fx 2 U : V (x) = tg. Then Wt is a smooth hypersurface, and Wt = t(W ). The tangent space TxWt of Wt at x = p(t) is equal to d t(p)TpW . We are now ready to show that dV ! on U . Pick x 2 U , w 2 TxM . Let t, p be such that x = p(t), p 2 W , so x 2 Wt. Let v 2 TpM be such that d t(p)v = w. De ne y(s) = d s(p)v, so y(s) 2 T p(s)Ws and y(t) = w. Then y satis es the variational equation _ y = @X @x ( p(s))y, and p satis es the adjoint variational equation _ p = @HX @x ( p(s)) = p @X @x ( p(s)). This implies that the function s ! h p(s); y(s)i is constant on (c "; c + "). If we choose w 2 TxWt, then v 2 TpW , so h p(c); vi = 0, and therefore h!(x); wi = h p(t); y(t)i = h p(c); y(c)i = 0 = hdV (x); wi, since V is constant on Wt. If we choose w = _ p(t), then v = _ p(c), so h!(x); wi = h p(t); y(t)i = h p(c); y(c)i = 1 = hdV (x); wi, since the derivative of V along p is obviously equal to 1. Since TxM is the direct sum of TxWt and the span of _ p(t), we conclude that !(x) = dV (x). Since ! dV and H(x; !(x)) 1 2, we have shown that V satis es the Hamilton-Jacobi equation H(x; dV (x)) = 1 2 or, equivalently, kdV (x)kG = 1. Now suppose that : [0; ] ! U is an E-arc parametrized by arc-length that goes from (t1) to (t2) for some t1; t2 in the interval (c "; c+ "). Then t2 t1 = V ( (t2)) V ( (t1)) = Z 0 dV ( (s)) ds ds = Z 0 hdV ( (s)); _ (s)ids Z 0 k _ (s)kGds = ; so the restriction of to any closed subinterval of (c "; c+") minimizes length in the class of U -valued E-curves. Now let be such that (i) 0 < < ", and (ii) the sub-Riemannian distance from ([c ; c+ ]) to the complement of U in M is larger than 2 . Then the restriction of to 92 WENSHENG LIU AND H ECTOR J. SUSSMANN the interval [c ; c+ ] minimizes length in the class of all M -valued E-curves. Appendix D: Rigid sub-Riemannian arcs and local optimality Rigid curves of rank-two distributions were studied by R. Bryant and L. Hsu in [7]. It is not hard to show that rigid curves necessarily are abnormal extremals. The purpose of this appendix is to discuss the relationship between optimality and rigidity. We will establish, by means of examples, the following facts: 1 A rigid curve need not be locally optimal. This disproves, at least for general |not necessarily regular| sub-Riemannian structures, a conjecture made by R. Montgomery in [19]. Our example involves a real-analytic sub-Riemannian structure in IR3, associated to a two-dimensional, non-regular, bracket-generating distribution, and the arc which is rigid but not locally optimal is real analytic, so we show in fact that rigidity does not imply local optimality even for real analytic curves. 2 An abnormal minimizer need not be rigid. This will be done by exhibiting, on the space IR4 endowed with a sub-Riemannian structure arising from a three-dimensional distribution, an abnormal sub-Riemannian minimizer that is not rigid. 3 Every abnormal extremal that can be put in the normal form of Theorem 4 is locally rigid. This yields in particular an alternative proof of the main theorem of Bryant and Hsu [7] |which says, in our terminology, that every regular abnormal extremal is locally rigid| for a slightly larger class of distributions (cf. Remark 2). Our examples show that, although all rigid sub-Riemannian curves SUB-RIEMANNIAN METRICS 93 are abnormal extremals, and the abnormal extremals studied here | i.e. regular abnormal extremals for two-dimensional distributions| are both minimizing and rigid, there is no general relation between the concepts of rigidity and optimality. We rst review the de nition of rigidity. LetM be an n-dimensional connected manifold and E be an m-dimensional distribution ofM . For any two points p; q in M , and any interval [a; b], let a;b E (p; q) be the space of C1 E-arcs : [a; b] ! M that satisfy (a) = p; (b) = q, endowed with the C1-topology. Let E(p; q) = [ 1 0, the function ' must vanish identically along . But this means that the curve ̂ : t ! (x(t); y(t)) is entirely contained in the zero set of ', which is the union of the straight line x = 0 and the parabola y = x2. Since ̂ goes from left to right (because v(t) > 0) and goes from ( L;L2) to (L;L2), the only SUB-RIEMANNIAN METRICS 95 possibility is that, up to a time reparametrization, ̂ coincides with the projection of on the (x; y) plane. But then and are equivalent modulo time reparametrization. So is rigid, as stated. We now prove that L is not time-optimal. We will construct, for each L > 0 and each " such that 0 < " L, a trajectory L;" that also goes from pL to qL, and we will show that L;" is shorter than L if " is small enough. We construct L;" by rst constructing a curve ̂L;" : t ! (x";L(t); y";L(t)) in IR2, de ned on an interval I"(L) = [ "(L); "(L)+4"̂], and parametrized by arc-length, that goes from ( L;L2) to (L;L2). Here the number "(L) will be de ned below, and "̂ is a positive number that will be chosen as a function of " and L. We then de ne the third component z";L : I"(L) ! IR by letting z(t) = R t (L) '(x";L(s); y";L(s))2v";L(s)ds, where v";L = _ x";L. The resulting curve L;" : t ! (x";L(t); y";L(t); z";L(t)) is then a trajectory of the control system that starts at pL and ends at a point q";L. If z";L( "(L)) = 0, then q";L is in fact qL. We construct ̂L;" in two stages. Starting from the plane curve ̂ : t ! (x (t); y (t)), de ned on I(L), we rst replace the portion that goes from ( "; "2) to ("; "2) by the straight-line segment " that joins these two points. This produces a slightly shorter curve, which we parameterize by arc-length, choosing the initial condition so that the curve goes through (0; "2) at time 0. Then the new curve is de ned on a time interval [ "(L); "(L)]. We then insert at the end a square S";L, oriented clockwise, whose sides are parallel to the coordinate axes and have length "̂. Moreover, we choose S";L so that its rst segment is horizontal and going from left to right. This implies that S";L lies below the curve y = x2. Then the total length new of the new curve is new = 2 "(L) + 4"̂. The length of the segment " is equal to 2", whereas the length of the piece replaced by " is 2" + 4 3"3 +O("5). So "(L) = (L) 23"3(1 + o(1)) as "! 0. Therefore new is equal to the length of L plus , where = 4 3"3(1 + o(1)) + 4"̂. 96 WENSHENG LIU AND H ECTOR J. SUSSMANN On the segment ", the inequalities jxj " and jy x2j "2 hold. Therefore '2 is bounded by "10. Since 0 < v";L(t) 1, we conclude that the integral of '(x";L; y";L)2v";L over ", which is equal to z";L( "(L)), is bounded by 2"11. We will choose "̂ as a function of " and L in such a way that "̂! 0 as "! 0. The nal value z";L( "(L)+4"̂) is equal to z";L( "(L))+RS";L '2dx. By Green's Theorem, the integral is equal |since S";L is oriented clockwise| to 2 R RR";L '@' @y dx dy, where R";L is the region enclosed by S";L. Since '(L;L2) = 0 and @' @y (L;L2) = L3 > 0, the double integral is negative, since S";L lies below the curve y = x2, so that ' is negative on R";L. Moreover, the Taylor expansion for ' near (L;L2) is '(x; y) = 2L4(x L) + L3(y L2) + higher order terms ; which implies that the double integral is equal to k"̂3(1 + o(1)) as "̂ ! 0, where k is a positive (L-dependent) constant. More precisely, the double integral is equal to k"̂3(1 + ("̂)), where ("̂) ! 0 as "̂! 0. We now choose "̂ so that k"̂3(1+ ("̂)) = z";L( "(L)). (The existence of "̂ is obvious. Indeed, the function ! k (1 + ( 1=3)) is of class C1 near = 0, and its derivative at = 0 is k. Hence, for every close to 0 there exists a unique ( ), also close to 0, such that ( )(1+ ( ( )1=3)) = . If we let "̂ = ̂( )1=3, with = z";L( "(L)), then "̂ satis es the desired equation.) Since z";L( "(L)) is bounded by 2"11, we conclude that "̂ is bounded by C" 11 3 , where C is an L-dependent positive constant. With our choice of "̂, the integral of '2dx over S";L exactly cancels that over ". So the resulting trajectory L;" goes from pL to qL. We now show that if " is small enough then L;" is shorter than L. Recall that the length of L;" is equal to the length of L plus , where = 43"3(1 + o(1)) + 4"̂. Since "̂ is bounded by C" 11 3 , and 11 3 > 3, we have < 0 if " is small enough, and our desired conclusion follows. SUB-RIEMANNIAN METRICS 97 Our next example shows that an abnormal minimizer need not be rigid in general. The example is essentially a modi cation of the one discussed in Section 2.3. Let (M;E;G) be the sub-Riemannian manifold of Section 2.3. Let M̂ = IR4 with coordinates (x1; : : : ; x4) and take Ê to be the kernel of the 1-form !̂ = x21dx2 (1 x1)dx3. Since !̂ never vanishes, Ê is a smooth 3-dimensional distribution. The three vector elds f̂ = @ @x1 ; ĝ = (1 x1) @ @x2 + x21 @ @x3 ; h = @ @x4 (76) form a global basis of sections of Ê. Clearly Ê is bracket-generating. Let Ĝ = dx21 + (x1)(dx22 + dx23) + dx24 ; where (x1) = 1 (1 x1)2 + x41 ; so f̂ , ĝ and h form an orthonormal basis of sections of Ê. Let (t) = (0; t; 0; 0) for t 2 IR. Then is Ê-admissible. From Proposition 1 we can conclude that the restriction d[a; b] is a minimizer if b a 2 3 . (To see this, let ~ : [ ; ] ! IR4 be another Ê-arc that goes from p = (0; a; 0; 0) to q = (0; b; 0; 0). Let , ~ be the projections of , ~ to the rst three coordinates. Then and ~ are E-arcs going from (0; a; 0) to (0; b; 0). Proposition 1 implies that is a minimizer for (M;E;G). So k kG k~ kG. Clearly, k~ kG k~ kĜ and k kG = k kĜ. So k kĜ k~ kĜ.) It is easy to see that is an abnormal minimizer. We show that is not rigid. For each integer j > 0, take a smooth real valued function j on [a; b] such that j(a) = j(b) = 0, and k _ jksup 1=j. Then the Ê-arc j given by j(t) = (0; t; 0; j(t)) for a t b goes from p to q. Clearly, j ! and _ j ! _ uniformly on [a; b], which shows that is not rigid. Finally we discuss the rigid curves of rank-two distributions. We prove: Theorem 14 Let E be a two-dimensional distribution on a manifold M . Then all the regular abnormal extremals of E are locally rigid. 98 WENSHENG LIU AND H ECTOR J. SUSSMANN PROOF. In view of Theorem 4, it su ces to prove Lemma 5 Let n 3, and let U be an open subset of IRn. Let ', 1; 2; 4; : : : ; n, 1; 3; : : : ; n be smooth functions on U , and let E be the distribution on U spanned by X = '(x) @ @x1 ; Y = @ @x2 + x1 n X k=1 k(x) @ @xk ; (77) where 3 = x1 1 + x3 3 + : : : + xn n. Let x : [a; b]! U be an E-arc given by x (t) = (0; x2 + t a; 0; : : : ; 0). Assume that 1(x (t)) 6= 0 for a t b. Then x is locally rigid. PROOF. Let K = fx (t) : a t bg, so K is a compact subset of U . Let V U be open, and such that K V and the closure V of V is compact and contained in U . We may assume that V is small enough so that 1(x) 6= 0 and 1+x1 2(x) > 0 on V . Without loss of generality we assume that 1(x) > 0 on V . Let C0 = inf n 1(x) 1 + x1 2(x) : x 2 V o ; C1 = sup n j i(x)j 1 + x1 2(x) : x 2 V; i = 3; : : : ; no ; C2 = sup n j i(x)j 1 + x1 2(x) : x 2 V; i = 3; : : : ; no : Let = min fkx yk : x 2 K; y 2 IRn V g. Then > 0. Let ~ = min n 2; 2C0 9nC1C2o. Now let a t1 t2 b be such that = t2 t1 < ~ . Let be the restriction of x to the interval [t1; t2]. We show that is rigid. Assume that : [t1; t2] ! U is an E-arc that goes from (t1) to (t2) and satis es k _ (t) _ (t)k < 1 2 for all t 2 [t1; t2]. Then k _ (t)k < 2, and therefore is contained in V . Write (t) = ( 1(t); : : : ; n(t)). Then we have 1 2 _ 2(t) 3 2 for all t 2 [t1; t2]. Now since is E-admissible, we have _ k(t) = 1(t) k( (t)) _ 2(t) 1 + 1(t) 2( (t)) SUB-RIEMANNIAN METRICS 99 for k 3. Therefore j k(t)j 3C2 2 R t2 t1 j 1(s)jds for k = 3; : : : ; n. Since 3( ) = 1 1( ) +Pnk=3 k k( ), we have 0 = 3(t2) 3(t1) = Z t2 t1 2 1(s) 1( (s)) _ 2(s) 1 + 1(s) 2( (s)) ds+ n X k=3 Z t2 t1 1(s) k(s) k( (s)) _ 2(s) 1 + 1(s) 2( (s)) ds C0 2 Z t2 t1 2 1(s)ds 9nC1C2 4 Z t2 t1 j 1(s)jds 2 2C0 9n C1C2 4 Z t2 t1 2 1(s)ds : Therefore 1 0, which implies that is a reparametrization of . Appendix E: A nonoptimality proof We prove that the nonsmooth abnormal extremal of Section 9.6 is not locally optimal. It su ces to show that for every L > 0 the restriction L of to [ L;L] is not optimal. In view of the symmetry of E and the metric with respect to the dilations Dr : (x1; x2; x3; x4; x5; x6) ! (rx1; rx2; r2x3; r3x4; r3x5; r4x6), it su ces to prove that 1 is not optimal. We let : [ 1; 1] ! IR2 denote the projection of 1 to the rst two coordinates. We will construct, for " > 0, " small, a polygonal curve " in IR2 going from (0; 1) to (1; 0) such that (i) " is shorter than , and (ii) " is the projection of an E-curve " in IR6 that goes from (0; 1; 0; 0; 0; 0) to (1; 0; 0; 0; 0; 0). Actually, " will also depend on a 5-tuple = ( 0; : : : ; 4) of positive numbers, which will later be chosen to depend on ", but will be o(") (in fact, O("4=3)) as " ! 0. To de ne the ", let a1; a2; a3; a4 be four xed numbers to be chosen later, such that 1 < a1 < 0, 1 < a2 < 0, 0 < a3 < 1, 0 < a4 < 1. Let a = min( a1; a2; a3; a4). For p1; : : : ; pn 2 IR2, use p1 ! p2 ! : : : ! pn to denote the polygonal curve obtained by concatenating the straight-line segments from p1 to p2, p2 to p3, : : :, pn 1 to pn. For " < 1 3 , de ne 1 " by modifying as follows: the portion 100 WENSHENG LIU AND H ECTOR J. SUSSMANN of that goes from (0; ") to (3"; 0) is replaced with the polygonal (0; ") ! ("; 0) ! ("; " 4) ! (3"; " 4) ! (3"; 0). Then observe that, if 3" < a, and we let p0 = ( " 2 ; " 2 ), pi = (0; ai) for i = 1; 2, pi = (ai; 0) for i = 3; 4, then 1 " goes through the ve points p0; : : : ; p4. We then de ne " as the modi cation of " obtained by inserting ve squares S0; : : : ; S4 at the points p0; : : : ; p4. Here Si is the square pi ! qi ! ri ! si ! pi where we let, using e1 = (1; 0), e2 = (0; 1): q0 = p0 + 0e1 ; r0 = q0 0e2 ; s0 = r0 1e1 ; q1 = p1 + 1e2 ; r1 = q1 1e1 ; s1 = r1 1e2 ; q2 = p2 2e1 ; r2 = q2 + 2e2 ; s2 = r2 + 2e1 ; q3 = p3 + 3e1 ; r3 = q3 + 3e2 ; s3 = r3 3e1 ; q4 = p4 + 4e2 ; r4 = q4 + 4e1 ; s4 = r4 4e2 : Then |writing j j = 0+: : :+ 4| " has length 2+4j j+(p2+ 12 2)". Since the i will be o("), this will guarantee that " is shorter than for small ". Now let " be the unique E-curve that projects down to " and starts at (0; 1; 0; 0; 0; 0). Let (1; 0; 1; 2; 3; 4) be the coordinates of the terminal point of ". Our desired conclusion will follow if we show that the functions i can be chosen so that 1 = : : : = 4 = 0. Expressing the i as functions of all six variables "; 0; : : : ; 4, an elementary computation shows that 1 = 2 1 2 2 + 2 3 2 4 2 0 ; (78) 2 = 2a3 2 3 2a4 2 4 + 2("; ) (79) 3 = 2a1 2 1 2a2 2 2 + 3("; ) (80) 4 = a1 3 1 + a2 3 2 + a3 3 3 a4 3 4 + "4 6 + 4("; ) ; (81) where the i are real-analytic functions that satisfy i("; ) = O("3 + " 2 0 + j j3) for i = 2; 3 ; (82) SUB-RIEMANNIAN METRICS1014("; ) = O(j j4 + "2 20)(83)as "; ! 0. Write " = 3, i = 4 i. Then the conditions 1 = : : : =4 = 0 are equivalent to2122 +2324 =20 ; (84)2a323 2a424 + 2( ; ) = 0 ;(85)2a121 2a222 + 3( ; ) = 0 ;(86)a131 + a232 + a333 a434 + 16 + 4( ; ) = 0 ;(87)where i( ; ) = 9 i( 3; ) for i = 2; 3, 4( ; ) = 13 ( 3; ). Thenthe i are still analytic near the origin, in view of (82) and (83). Fixthe value of 4, say by taking 4 = 1. For ( 1; 2; 3) in the setC = f(x1; x2; x3) :x21+ x23 > 1 +x22g, de ne 0 as an analytic functionof ( 1; 2; 3) with positive values, via (84). Then the left-hand sidesof (85), (86), (87) are analytic functions of 1; 2; 3, and . Picka solution (
1;
2;
3) 2 C of (85), (86), (87) for = 0, such that1 6=
2. (For example, we may choose a1 = 13 , a2 = 112, a3 = 1108,a4 = 112,
1 = 1,
2 = 2,
3 = 3.) The Jacobian determinant withrespect to ( 1; 2; 3) of the left-hand sides of (85), (86), (87) is equal,at (
1;
2;
3), to 48a1a2a3
1
2
3(1
2), which is 6= 0. Then theImplicit Function Theorem implies that the system (85), (86), (87) |with 4 1, 0 > 0 de ned by (84), and = 4 | has, for small, a solution ( 1( ); 2( ); 3( )) that depends analytically on andis equal to (
1;
2;
3) for = 0. Letting i(") = "4=3 i("1=3), all ourrequirements are satis ed. 102WENSHENG LIU AND H ECTOR J. SUSSMANNReferences[1] C. Bar, Carnot-Caratheodory Metriken, Diplomarbeit, Bonn, 1988.[2] L.D. Berkovitz, Optimal Control Theory, Springer-Verlag, NewYork, 1974.[3] J.M. Bismut, Large Deviations and the Malliavin Calculus,Birkhauser, 1984.[4] V.G. Boltyanskii, \Su cient conditions for optimality and the jus-ti cation of the Dynamic Programming Principle," SIAM J. Con-trol 4 (1966), pp. 326-361.[5] R.W. Brockett, \Control Theory and Singular Riemannian Geom-etry," in New Directions in Applied Mathematics (P.J. Hilton andG.S. Young, eds.), Springer-Verlag (1981), pp. 11-27.[6] R.W. Brockett, \Nonlinear Control Theory and Di erential Ge-ometry," in Proc. 1983 Int. Congress of Mathematicians, Vol. 2,North-Holland (1984), pp. 1357-1368.[7] R. Bryant, L. Hsu, \Rigidity of Integral Curves of Rank 2 Distri-butions," Inventiones Mathematicae 114 (1993), pp. 435-461.[8] W.L. Chow, \Uber Systeme Von Linearen Partiellen Di erential-gleichungen Erster Ordnung", Math. Ann. 117 (1940-1941), pp.98-105.[9] B. Gaveau, \Principe de moindre action, propagation de la chaleuret estimees sous elliptiques sur certains groupes nilpotents," ActaMath. 139 (1977), pp. 94-153.[10] Z. Ge, \Horizontal Path Spaces and Carnot-Caratheodory Met-rics," Paci c Journal of Mathematics 161, No. 2 (1993), pp. 255-286.[11] Z. Ge, \On the Global Geometry of Sub-Riemannian 3-Manifolds,I," preprint. SUB-RIEMANNIAN METRICS103[12] U. Hamenstadt, \Some Regularity Theorems for Carnot-Caratheodory Metrics," J. of Di . Geometry 32 (1990), pp. 819-850.[13] B. Kaskosz and S. Lojasiewicz Jr., \A Maximum Principle forgeneralized control systems," Nonlinear Anal. TMA 9 (1985), pp.109-130.[14] I. Kupka, \Abnormal extremals," preprint, 1992.[15] E.B. Lee and L. Markus, Foundations of Optimal Control Theory,Wiley, New York, 1968.[16] W.S. Liu and H.J. Sussmann, \Abnormal Sub-Riemannian Mini-mizers," in Di erential Equations, Dynamical Systems and ControlScience, K.D. Elworthy, W.N. Everitt, and E.B. Lee Eds., LectureNotes in Pure and Applied Mathematics, Vol. 152. M. Dekker,New York, 1993. Pages 705 716. (IMA technical report #1059,November, 1992.)[17] R. Montgomery, \Abnormal Minimizers," to appear in SIAM J.Control and Opt.[18] R. Montgomery, \Characteristics, Singularities, and Abnormals forSystems Linear in Controls," 1993 preprint.[19] R. Montgomery, \Survey of Singular Geodesics," 1992 preprint.[20] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F.Mischenko, The Mathematical Theory of Optimal Processes, Wiley,New York, 1962.[21] E.D. Sontag, \Universal nonsingular controls," Systems and Con-trol Letters 19 (1992), pp. 221-224.[22] R. Strichartz, \Sub-Riemannian Geometry," J. Di . Geom. 24(1986), pp. 221-263.[23] R. Strichartz, \Corrections to `Sub-Riemannian Geometry'," J.Di . Geom. 30 (1989), pp. 595-596.[24] H. J. Sussmann, \Orbits of families of vector elds and integra-bility of distributions," Trans. Amer. Math. Soc. 180 (1973), pp.171-188. 104WENSHENG LIU AND H ECTOR J. SUSSMANN[25] H. J. Sussmann, \A Cornucopia of Abnormal Sub-RiemannianMinimizers. Part I: The Four-Dimensional Case," IMA technicalreport #1073, December 1992.[26] H. J. Sussmann, \An Introduction to the Coordinate-Free Maxi-mum Principle," 1993 preprint.[27] T. J. Taylor, \Some Aspects of Di erential Geometry Associatedwith Hypoelliptic Second Order Operators," Paci c Journal ofMathematics 136, No. 2 (1989) pp. 355-378.[28] T. J. Taylor, \O Diagonal Asymptotics of Hypoelliptic Di usionEquations and Singular Riemannian Geometric," Paci c Journalof Mathematics 136, No. 2 (1989) pp. 379-399.[29] M. Zhitomirskii, Typical Singularities of Di erential 1-Forms andPfa an Equations, American Mathematical Society Translationsof Mathematical Monographs, Vol. 113, 1992.Wensheng LiuDepartment of MathematicsRutgers UniversityNew Brunswick, NJ 08903E-mail: [email protected] J. SussmannDepartment of MathematicsRutgers UniversityNew Brunswick, NJ 08903E-mail: [email protected]